1102
Daily Python News Question, Tips and Tricks, Best Practices on Python Programming Language Find more reddit channels over at @r_channels
Real-time YouTube Comment Sentiment Analysis with Kafka, Spark, Docker, and Streamlit
Hey r/Python! 👋
# What My Project Does:
This project performs real-time sentiment analysis on YouTube comments using a stack of Kafka, Spark, Docker, and Streamlit. It classifies comments into positive, neutral, or negative sentiments and displays the results in a web interface for easy visualization and interpretation. The aim is to provide insights on how users are reacting to YouTube videos in real-time, which can be especially useful for content creators, marketers, or analysts who want to track audience reception.
# Target Audience:
This project is primarily a learning-focused, proof-of-concept to demonstrate the power of real-time big data analytics with modern tools. While it could potentially be expanded into a production-ready system, it’s currently a toy project meant for educational purposes and exploring various technologies. Developers looking to explore Kafka, Spark, and Streamlit in a Dockerized environment will find this project helpful.
# Comparison:
What sets this project apart from existing alternatives is its real-time processing capability combined with the use of big data tools. Most sentiment analysis projects process data in batch mode or on a smaller scale, while this project uses Kafka for real-time streaming and Spark for distributed processing. It’s also containerized with Docker, which makes it easy to deploy and scale.
/r/Python
https://redd.it/1g7cs5l
FriendlyDateParser, just another Python module for date parsing!
Hello!
I've just released **FriendlyDateParser**, just another Python module for date parsing.
## What My Project Does
It can parse complex date expressions like:
* `2 days before the last day of next month`
* `1h15m after next Sunday at midnight CEST`
* `the second Monday of 2012`
The goal is to make working with date references straightforward, even when the expressions are complex.
## Target Audience
At this point, the module is still young and bugs should be expected. There may still be some edge cases which are not handled correctly.
## Comparison
Well, actually the reason for writing this module is that I had been using [**dateparser**](https://dateparser.readthedocs.io/) (*the* mature module with a similar purpose) for a while, but I found it was not able to handle all the cases I needed. So, I created **FriendlyDateParser** to address those gaps.
On the other hand, `dateparser` is multilingual while `friendlydateparser` only support English expressions (and I don't plan to extend the module in that way).
## Links
Check out the documentation and repo [here](https://github.com/salva/py-friendlydateparser)!
The module is also available from [PyPI](https://pypi.org/project/friendlydateparser/)
I'd love to hear your feedback and see how it works for you!
/r/Python
https://redd.it/1g6nwca
filefrag - library and executable to explore file fragmentation
Spent last night making this, added some turd polish today and added it to pypi.
## 🤷 why/what?
I wanted to get file fragmentation info so I can punch holes in files, aligned with memory pages. But I really didn't want to parse filefrag's outputs, so I wrote a python version with a friendly API and a command line that can produce json.
It only works on Linux as it depends on the FIE interface, but pull requests welcome etc.
## ⚒️ how?
See the video for a demo including installing from source, but you can install with pip:pip install filefrag
Then you can run pyfilefrag, see --help for details. It has --verbose, and --json outputs for your parsing pleasure.
To use the library, just call filefrag.FileMap('/path/whatever') to build a map of the extents in the file using ioctl's interface. Then you can poke about in the guts of a file:
⛓️💥 inspect fragmentation
🔍 find out where data is on your physical drive
🟰 compare extents between paths
📔 use them as dict keys
🕳️ check files for holes, like before and after hole punching
✅verify your XFS deduplication strategy, write your own stats tool
💩 dump file layouts to json (`print(f"{filemap:j}"`)
⚠️ break your disk
/r/Python
https://redd.it/1g70lqb
Best resources to learn
Having always use node js for my backends, I’m trying to widen my skills. What are the go to resources to learn Django?
/r/djangolearning
https://redd.it/1g6ull9
stick or move to another
So i made an inventory management system in Django for my internship project.
i chose Django for several reasons:
- i wanted to learn python and discover how to do weB apps with it
- i wanted to be unique because all of my colleagues used PHP (Laravel)
- Django has pretty much everything compared to Flask
- ...
Django is the first backend web framework i ever tried.
About the experience..., it was not that good i don't why.
Is it bc the result code IT WAS LITERALLY A MESS
i could not fix bugs or add features without breaking something
and then rollback to the previous version.
and sometimes i wanted to something simple but Django make it hard
like custom forms and how to integrate Django with CSS frameworks make it
an absolute MESS.
what i am missing in my knowledge to have a great experience with Django?
do i miss something in Django or in web/dev itself?
did you have some issues too with Django at your early days learning it?
should i move to something else like Laravel or Spring (ik some Java and PHP)
or just stick with Django?
bc i feel like it is not for me.
project repo
i hope i explained the problem.
thank you in advance.
/r/django
https://redd.it/1g6q4h5
Saturday Daily Thread: Resource Request and Sharing! Daily Thread
# Weekly Thread: Resource Request and Sharing 📚
Stumbled upon a useful Python resource? Or are you looking for a guide on a specific topic? Welcome to the Resource Request and Sharing thread!
## How it Works:
1. Request: Can't find a resource on a particular topic? Ask here!
2. Share: Found something useful? Share it with the community.
3. Review: Give or get opinions on Python resources you've used.
## Guidelines:
Please include the type of resource (e.g., book, video, article) and the topic.
Always be respectful when reviewing someone else's shared resource.
## Example Shares:
1. Book: "Fluent Python" \- Great for understanding Pythonic idioms.
2. Video: Python Data Structures \- Excellent overview of Python's built-in data structures.
3. Article: Understanding Python Decorators \- A deep dive into decorators.
## Example Requests:
1. Looking for: Video tutorials on web scraping with Python.
2. Need: Book recommendations for Python machine learning.
Share the knowledge, enrich the community. Happy learning! 🌟
/r/Python
https://redd.it/1g6wbs0
PyQt best option for commercial use?
I'm looking to possibly develop an app that will run on a Linux Desktop, specifically Ubuntu, and the latest OS X. The UI and performance are very important. Is PyQt my best option?
/r/Python
https://redd.it/1g6brra
I built an open-source AI-driven Code Review app for GitHub repos
What My Project Does
Hi Everyone,
I recently built an open-source GitHub app in Django/python that can post a detailed line-by-line code review on any new PR. I'd love help in testing it as I seek feedback on it.
Here is the app: https://gitpack.co/
Here is the source-code: https://github.com/gitpack-ai/gitpack-ai and an example PR review: https://github.com/gitpack-ai/gitpack-ai/pull/9
It's free for open-source repos, but I can enable this for private repos for a month or so, if you DM me. Appreciate your feedback! I hope you all can find value in it.
Target Audience
Anyone who is actively developing on GitHub
/r/Python
https://redd.it/1g6m2mb
In-depth Django + Celery tutorial
https://www.youtube.com/watch?v=RY74ug36KUc
/r/djangolearning
https://redd.it/1g6eiwg
Chat System/Room
Trying to build a chat system where a user can build a chat room and invite some users to chat amongst them.
I have never built a chat system before.Also,this is a backend api project where I'm building api for the frontend.Where and how should I start.
I have built some api and common apps with django and django rest framework.I have the general idea.
/r/django
https://redd.it/1g6fk0a
Best way to upload file in Django
What's difference using Django UploadFileForm described at https://docs.djangoproject.com/en/5.1/topics/http/file-uploads/
from FileUploadView APIView?
Why should it be serialized? And how is the file is serialized?
Why doesn't it just dump the binary on disk?
It looks like they're making simple thing very hard.
/r/django
https://redd.it/1g6c4a7
I Built a Django Package for Google Analytics Integration!
Hey everyone!
I created a Django package that makes it super easy to integrate Google Analytics GA4 into your projects. Here are some features:
Supports Universal Analytics & GA4
IP anonymization and cookie settings
Server-side tracking via middleware
Debug mode for dev environments
Event tracking & custom dimensions
Excludes staff users from tracking
Check it out here: PyPI 👈 github
Contributions are welcome on GitHub! Let me know what you think! 😄
/r/djangolearning
https://redd.it/1g5nb0y
Friday Daily Thread: r/Python Meta and Free-Talk Fridays
# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️
Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!
## How it Works:
1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.
## Guidelines:
All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.
## Example Topics:
1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.
Let's keep the conversation going. Happy discussing! 🌟
/r/Python
https://redd.it/1g654oy
I've recently applied to a mid-level position and would like to know if it is normal to ask for all of this as a technical task for a company
code### Technical Assessment: Full-Stack Application Development with Django, React, and AWS
As part of the interview process, candidates are required to complete a technical assessment. The task involves developing a secure full-stack application with a Django backend, React frontend, and a PostgreSQL serverless database hosted on AWS Aurora RDS. The application must adhere to HIPAA compliance and include audit logging for security and monitoring purposes. Candidates are expected to containerize the application using Docker and deploy it to AWS using their own AWS account. Below are the detailed requirements for the assessment.
# Key Requirements
# General Requirements:
Tech Stack:
Backend: Python, Django (Rest Framework)
Frontend: React (with functional components and hooks)
Database: PostgreSQL (AWS Aurora Serverless)
Containerization: Docker
Deployment: AWS (EC2 or ECS with Copilot, using RDS Aurora for the database)
Source Code Management:
The code should be hosted on a GitHub repository, with a link provided.
Use Git for version control, following best practices with regular, well-documented commits.
Deployment:
Candidates should provide a live link to the deployed app running on AWS. The application must be deployed using Docker and hosted on AWS using an Aurora Serverless PostgreSQL instance for the database.
The
/r/django
https://redd.it/1g60ggm
Advanced python tips, libraries or best practices from experts?
I have been working as a software engineer for about 2 years and python was always my go to language while building various different application. I always tried to keep my code clean and implement best practices as much as possible.
I wonder if there are many more tips which could enhance the way I write python?
/r/Python
https://redd.it/1g5xswk
Creating simple inventory management app
Hi all, I'm trying to learn about Flask and decided to create a simple inventory management app which allows me to add, remove and edit entries in sqlite db using frontend. I was able to make the python app work but I'm stuck on the frontend part. I have the html file in "templates" folder and the the js script in the "static" folder, but when I try to run it, I end up with this error: "Failed to load resource: the server responded with a status of 404 (NOT FOUND)". Can someone help me out on what I'm missing?
Here's my repo: https://github.com/iraklikeshelava/inventory-management
/r/flask
https://redd.it/1g6fcyb
How to write the Terms and privacy section of the website ?
i got to that point where you must write the terms and privacy, and i was thinking if there is a custom terms or privacy to include or rules to follow when writing them, any suggestions ?
/r/djangolearning
https://redd.it/1g765b5
Django SQL Lite DB Built-in Size !?
I want to create a django app but for now i use a script on my PC the fetch some data and all the final results are stored into a json that can reach size of 600 to 700 MBs.... Can the DB store this size ? I have a hosting provider where to put the app on and unfortunate, my current plan doesn't have access to bigger DBs, only some that can store up to 350 MBs...
/r/django
https://redd.it/1g7022w
Aid with using anaconda to load fashionmnist
Hello could someone please step out how from the anaconda.navigator to load fashionmnist (which is on a laptop that’s never run it and may be missing things needed)
/r/Python
https://redd.it/1g6wjqj
My first full stack app
https://redd.it/1g6uq6p
@pythondaily
Lost my sqlite database😱. After that I wrote a backup script that runs once a day
This is just a warning to all you guys, remember to backup your database cause shit happens. Now I rest easy cause I wrote a backup script that runs automatically once a day just to backup my db, I know its going to fill up my space very soon and raise my costs but I do think its worth it, by the way the project I lost the db for was still a new project so there was not a lot of records in there anyway so not all hope is lost
/r/django
https://redd.it/1g6l5dv
I migrated from digital ocean to hetzner - (good experience)
I am building a streaming platform for my wife.
I tried to find the best option over all the cloud providers. At first I decided to use Digital Ocean because of their user friendly docs and UI. Also finding terraform docs was very easy. To be honest setting up the project was very easy so I can say I am satisfied with that.
However the outbound limit of 4TB, was a deal breaker for me. At the beginning I was thinking the limitting the video quality at 720P, however my wife wanted to have 1080P.
I started my search and find out that, hetzner was not only offering the cheap hardware but also the traffic limits were way better. To compare the numbers:
digital ocean: 24$
- 80gb ssd,
- 4gb ram
- 2Vcpu
- 4TB outbound
hetzner: 16€
- 160gb ssd
- 8gb ram
- 4Vcpu
- 20TB outbound.
So I decided to deploy my project on their servers. My concern was not finding good terraform docs for that, but tbh it was very easy and straight forward. In less then 3 hours I was able to run my
/r/django
https://redd.it/1g6kive
PyTraceToIX - Debugging Jinja2 template, Flask web apps without breaking the design or code changes
Project on GitHub
## What My Project Does
PyTraceToIX is an expression tracer designed for debugging Jinja2 templates, Flask web apps, lambdas, list comprehensions, method chaining, and expressions in general.
Code editors often cannot set breakpoints within these kinds of expressions, which requires significant code modifications to debug effectively.
For Jinja2 templates, the debug extension can be used, but it typically dumps the entire context, making it difficult to isolate specific issues.
PyTraceToIX solves this by allowing developers to trace and write specific data directly to sys.stdout or a stream without altering the design or making any changes to the web application.
Additionally, PyTraceToIX can capture multiple inputs and their results, displaying them all in a single line, making it easier to view aggregated data and trace the flow of values.
PyTraceToIX offers a straightforward solution to these challenges, simplifying debugging while preserving the integrity of the original codebase.
It was designed to be simple, with easily identifiable functions that can be removed once the bug is found.
PyTraceToIX has 2 major functions:
- c capture the input of an expression input. ex: c(x)
- d display the result of an expression and all the captured inputs. ex: d(c(x) + c(y))
And 2 optional functions:
- init initializes display format, output stream and
/r/Python
https://redd.it/1g6fd28
Feature Friday: Model Choices!
Time for another Django Feature Friday! 🚀
Django 5.0 introduced more options for declaring field choices. Now you can use Mapping, a callable, and strings for single-character choices, in addition to the traditional tuples and enumeration.
Previously, Field choices were limited to list of 2-tuples, or an Enumeration types subclass. Now, you can use:
* Strings (for single-character choices) without .choices
* Mapping instead of list of 2-tuples (with hierarchies)
* A callable instead of iterable
Here's an example showcasing these new options:
from django.db import models
Medal = models.TextChoices("Medal", "GOLD SILVER BRONZE")
SPORT_CHOICES = { # Using a mapping instead of a list of 2-tuples.
"Martial Arts": {"judo": "Judo", "karate": "Karate"},
"Racket": {"badminton": "Badminton", "tennis": "Tennis"},
"unknown": "Unknown",
}
def get_scores():
return [(i, str(i)) for i in range(10)]
class Winner(models.Model):
/r/django
https://redd.it/1g6grym
Django + Celery Tutorial
Hey, all!
I've made a text + video version of Celery tutorial.
Video: https://www.youtube.com/watch?v=RY74ug36KUc
Text: https://appliku.com/celery
This tutorial aims at beginners who struggle with understand what Celery is and how to use it and never set it up before.
I tried to do my best explaining use the concept of it, use cases + step by step instructions on setting Celery app.
The last bit is a real world example of a generating reports using Celery tasks.
Let me know what you think and I hope it helps at least few people to start using this powerful library!
/r/django
https://redd.it/1g6ela0
Should I extend Django's roles and permissions, or should I create my own roles and permissions layer?
Hello everybody, I want to develop a project management web app as a learning experience, and I have thought about the functionality of the roles and permissions for the users of the web app. Should I extend the Django system and not get too complicated, or should I keep the project roles separate? And about the users? I have also thought about adding specific things like: "You can mark a task as completed", "You can see all the team's tasks" etc. Thank you in advance for taking the time to read and leave your recommendations.
/r/django
https://redd.it/1g69h7g
Meta Python 3.13 is not compatible with asyncpg (fastest async library to access PostgreSQL)
Just a heads up. I was going to upgrade my projects today, but asyncpg won't install. I tried both docker (all slims and general one) and local (mac, though) – no difference.
UPD:
This is a known issue by asyncpg developer. I hope they roll out an update soon.
/r/Python
https://redd.it/1g5qo5p
I created an app to animate stock performance
https://reddit.com/link/1g616sq/video/peq1orw0qdvd1/player
A few weeks ago, I saw a post that shows a screen recording of their Robinhood account. The pnl movement animation felt more engaging than a static chart, and it really stood out for me.
So I built a tool to animate stock performance chart: animatestock.com
This simple app basically animates data in a line chart. It also gives you flexibility in customizing the chart to your liking. You can also use it for things like net worth, savings, or even # of your social media followers, etc.
Let me know if you find it useful in anyway. Appreciate it!
/r/flask
https://redd.it/1g616sq
On a scale of 1 to 10, how would you recommed Django to a beginner?
As someone is trying to learn python for the first time, would you guys recommend to learn python and django for the first time or learn any other like javascript or nodeJs. So far I'm comfortable with the javascript basics but I'm trying to learn python and django. Any advice from the experienced developers here for a beginner is really helpful. And also what do you think about django for the upcoming years of development.
Is there any roadmap to get a proper knowledge so you don't miss the most important things to learn?
/r/django
https://redd.it/1g5tggk
Project I was tired of reading through thousands of lines of documentation
Alongside Python, Im learning C. So as a way of testing how much I have learned, I decided to dabble in some socket programming in C. Found beej's guide, and got to work.
Problem? I hate reading too much educational stuff. I realized that after every 10 or so minutes I'd lose focus of what I was reading
Solution? Build a program into which I can stuff the ENTIRE documentation, then ask it questions, and it'll give me answers from the documentation I stuffed it with.
Behold, FTHEDOCS!
What my project does: It basically gives you a question-answer like interface to search the documentation, so you dont have to spend hours looking for those 2 lines.
Target audience: Anyone looking for a nicer way to read the docs. Or anyone who has a bunch of text and would like to search through it.
Comparisons: Not that I know of. Though I guess the Ctrl+F shortcut is kind of similiar
REPO: **https://github.com/muaaz-ur-habibi/fthedocs**
Do note: This was a fun project I built as a way to learn RAG, and to suite my specific needs. As a result, it might not be suited for you, though I tried my best to make it as customizable as possible.
Thanks
/r/Python
https://redd.it/1g5nyng