1102
Daily Python News Question, Tips and Tricks, Best Practices on Python Programming Language Find more reddit channels over at @r_channels
MetaDataScraper: A Python Package for scraping Facebook page data with ease!
Hey everyone! 👋
I’m excited to introduce MetaDataScraper, a Python package designed to automate the extraction of valuable data from Facebook pages. Whether you're tracking follower counts, post interactions, or multimedia content like videos, this tool makes scraping Facebook page data a breeze. No API keys or tedious manual effort required — just pure automation! 😎
Usage docs here at ReadTheDocs.
# Key Features:
Automated Extraction: Instantly fetch follower counts, post texts, likes, shares, and video links from public Facebook pages.
Comprehensive Data Retrieval: Get detailed insights from posts, including text content, interactions (likes, shares), and multimedia (videos, reels, etc.).
Loginless Scraping: With the LoginlessScraper class, no Facebook login is needed. Perfect for scraping public pages.
Logged-In Scraping: The LoggedInScraper class allows you to login to Facebook and bypass the limitations of loginless scraping. Access more content and private posts if needed.
Headless Operation: Scrapes data silently in the background (without opening a visible browser window) — perfect for automated tasks or server environments.
Flexible & Easy-to-Use: Simple setup, clear method calls, and works seamlessly with Selenium WebDriver.
# Example Usage:
1. Installation: Simply install via pip:
​
pip install MetaDataScraper
2) Loginless Scraping (no Facebook login required):
from MetaDataScraper import
/r/Python
https://redd.it/1gwn9yd
How I Escaped Python Dependency Hell with pip-tools
erik_48905/escape-python-dependency-hell-with-just-2-commands-3245fb0c0bb8" rel="nofollow">https://medium.com/@erik_48905/escape-python-dependency-hell-with-just-2-commands-3245fb0c0bb8
/r/flask
https://redd.it/1gwq6oj
Django Developers Survey 2024
https://jb.gg/asjljo
/r/django
https://redd.it/1gwkewu
R BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
Tired of saturated benchmarks? Want scope for a significant leap in capabilities?
Introducing BALROG: a Benchmark for Agentic LLM and VLM Reasoning On Games!
BALROG is a challenging benchmark for LLM agentic capabilities, designed to stay relevant for years to come.
Check it out!
GitHub: https://github.com/balrog-ai/BALROG
Leaderboard: https://balrogai.com
Paper: https://arxiv.org/abs/2411.13543
/r/MachineLearning
https://redd.it/1gwhnf8
Creating a Python System to Turn All PostgreSQL Servers into Masters with Auto-Recovery and Sync – N
Hello Python community!I’m currently working on developing a distributed PostgreSQL system using Python, where all servers act as masters. Additionally, I’m adopting a clear separation between servers and clients to create a flexible and efficient architecture.The primary goals of this project are as follows:
1. Master-Master architecture
All servers operate equally, eliminating single points of failure (SPOF).
2. Server-Client separation
Clients can seamlessly access the system while the internal operations are optimized for distributed workloads.
3. Automatic recovery
In case of server failures, other nodes automatically handle recovery to maintain uninterrupted service.
4. Automatic data synchronization
Efficiently synchronizing data across nodes while ensuring consistency.
5. Leveraging Python and PostgreSQL
Combining Python's flexibility with PostgreSQL's robust features.
# Current Tools
For this project, I’m focusing on the following two key modules:
psycopg3: To enable efficient communication with PostgreSQL, especially with its asynchronous capabilities.
aioquic: For leveraging the QUIC protocol to achieve fast and reliable data synchronization, particularly for server-client communications in a distributed setup.
# Challenges and Feedback Needed
Here are some specific points where I’d love to get your insights:
1. Server-Client Design Approach
What’s the best way to dynamically determine which server the client should
/r/Python
https://redd.it/1gwghji
SQLAlchemy Foreign Key Error: "Could not find table 'user' for assignment_reminder.teacher_id"
# Body:
# Problem Description:
I'm encountering an error when running my Flask application. The error occurs when I try to log in, and it seems related to the `AssignmentReminder` model's foreign key referencing the `User` model. Here's the error traceback:
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'assignment_reminder.teacher_id' could not find table 'user' with which to generate a foreign key to target column 'id'
# Relevant Code:
Here are the models involved:
**User Model**:
class User(db.Model, UserMixin):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(150), nullable=False, unique=True)
email = db.Column(db.String(120), unique=True, nullable=False)
password_hash = db.Column(db.String(128), nullable=False)
role = db.Column(db.String(20), nullable=False) # e.g., 'student', 'teacher', etc.
def __repr__(self):
return f"User('{self.username}', '{self.email}', '{self.role}')"
**AssignmentReminder Model**:
class AssignmentReminder(db.Model):
__tablename__ = 'assignment_reminder'
/r/flask
https://redd.it/1gwed4y
Tips to morph Internal DRF App into Multi Tenant SaaS Setup
Dear community,
I learned A TON over the last months from all the posts and great answers here.
My team and I are transforming our Django-React application (utilizing DRF, PostgreSQL, Redis, Celery, and MinIO) into a multi-tenant SaaS platform. Currently, the app relies heavily on manual admin management. Our goal is to streamline client onboarding and empower tenant admins to manage their settings, data, and users independently.
Points where we catch strays :
1. Multi-Tenancy Design: We're debating between using separate schemas or databases. We aim for robust data isolation but are concerned about the complexities of schema migrations and managing multiple databases.
2. Customization: Tenants wish to define custom fields (likely using JSON) and workflows. How can we efficiently handle queries and searches across these fields without overloading PostgreSQL?
3. Global Search: Tenants require robust search functionality, including full-text and nested searches. While Elasticsearch seems suitable, syncing tenant data dynamically poses challenges. In the interim, could PostgreSQL Views facilitate searches across models and relationships?
4. Throttling & Performance: How can we prevent resource hogging (e.g., database queries, Celery tasks) by "noisy neighbors" without complicating resource allocation?
5. Auth/Compliance: Supporting SSO and maintaining tenant-specific audit logs is becoming complex. Any advice on keeping this
/r/django
https://redd.it/1gwaw56
Should I Learn a New Tech or Start Applying?
Hello folks,
I've been working with Django for the past 3 months and have hands-on experience in Machine Learning, Computer Vision, and other AI-related projects. I'm pretty confident in Python and have completed two remote internships, each lasting 2 months.
I'm aiming for a decent package of around 5-6 LPA, but I'm at a crossroads:
1)Should I learn a different technology (like Node.js, since many job postings mention it), or is Django enough?
2)Should I start applying for jobs now or focus on adding more skills to match industry demands?
Also, can you suggest platforms or places where I can find Django-related job opportunities? Most openings I come across seem to require JavaScript or Node.js expertise.
Thank you in advance for your advice!
/r/django
https://redd.it/1gw81a7
Thursday Daily Thread: Python Careers, Courses, and Furthering Education!
# Weekly Thread: Professional Use, Jobs, and Education 🏢
Welcome to this week's discussion on Python in the professional world! This is your spot to talk about job hunting, career growth, and educational resources in Python. Please note, this thread is not for recruitment.
---
## How it Works:
1. Career Talk: Discuss using Python in your job, or the job market for Python roles.
2. Education Q&A: Ask or answer questions about Python courses, certifications, and educational resources.
3. Workplace Chat: Share your experiences, challenges, or success stories about using Python professionally.
---
## Guidelines:
- This thread is not for recruitment. For job postings, please see r/PythonJobs or the recruitment thread in the sidebar.
- Keep discussions relevant to Python in the professional and educational context.
---
## Example Topics:
1. Career Paths: What kinds of roles are out there for Python developers?
2. Certifications: Are Python certifications worth it?
3. Course Recommendations: Any good advanced Python courses to recommend?
4. Workplace Tools: What Python libraries are indispensable in your professional work?
5. Interview Tips: What types of Python questions are commonly asked in interviews?
---
Let's help each other grow in our careers and education. Happy discussing! 🌟
/r/Python
https://redd.it/1gw2e4u
Spelunking in Comments and Documentation for Security Footguns
Hi everyone, we just posted a new article on interesting security footguns that could pop up in applications using third-party Elixir, Python, and Golang libraries. It's a fast read, so check it out! https://blog.includesecurity.com/2024/11/spelunking-in-comments-and-documentation-for-security-footguns/
/r/Python
https://redd.it/1gvxlrz
Running 24/7 chromedriver python script
Hello guys,
I was wondering if its possible to run a python script using a chromedriver on a AWS script service, or do I need to use a VPS ? This is simple task, no huge traitment.
Thanks !
/r/Python
https://redd.it/1gvq8z2
Migrating from black and flake8 to ruff
as the title says, so i'm currently working on a relatively huge python/django codebase, built over the course of 6 years, which has been using black and flake8 for formatting and linting in pre-commit hook, both have their versions unupdated for about 3 years, now i have a somewhat difficult task on hand.
the formatting and linting engine is to be moved to ruff but in such a way that the formatting and linting changes reflected in codebase due to ruff are minimal, i can't seem to figure out a way of exporting either configs from black and flake8 in their current state so i can somehow replicate them in ruff to control the changes due to formatting. if anyone has been in a similar situation or know any potential way i can approach this, that would greatly help. cheers!
pre-commit-config.yaml (in its current state, as you can see versions are a bit older)
repos:
- repo: https://github.com/psf/black
rev: 19.10b0
hooks:
- id: black
/r/Python
https://redd.it/1gvnfvi
Boiler plate for flask back end.
I am looking for Flask boiler plate for creating backend services. I am exploring cookie cutter but it seems old. Is any other updated resource available ?
/r/Python
https://redd.it/1gvjp8p
I am thinking of making a tool script that makes it easier for linux admins
To many commands is a pain to recall the switches and so on. Anyone know of any shell script that can provide a menu then sub-menu of options?
/r/Python
https://redd.it/1gvhb2h
Has anyone here built a profitable side project with Django or created one for a client that generates profit?
Hi everyone,
I’m curious to hear from those who’ve created side projects using Django. Have any of you built something that turned out to be profitable, either as a personal project or for a client?
I’m working on a side project myself using Django and DRF, mainly focusing on the backend. While I enjoy the process, I’m also wondering about the potential for turning it into something financially viable.
Thanks in advance for sharing your experiences! I’m hoping this inspires ideas and helps me (and others) approach these projects with a more practical perspective.
/r/django
https://redd.it/1gv7q2v
Friday Daily Thread: r/Python Meta and Free-Talk Fridays
# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️
Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!
## How it Works:
1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.
## Guidelines:
All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.
## Example Topics:
1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.
Let's keep the conversation going. Happy discussing! 🌟
/r/Python
https://redd.it/1gwub4n
Finished my first Django app! (But deployment is hell)
I just finished my first django app. A simple crm for my company. Developing it was an experience that makes me want to switch carrers into web app development. It’s been really really awesome. Sadly I can’t say the same thing about deploying the app. I’ve been trying to get it to work on and off without complete success.
This is how my process looks like:
Pull from repo -> break gunicorn in various ways and spend half and hour figuring out what broke-> get asked to change something -> have fun modifying stuff in my development environment -> pull from repo -> break gunicorn in various ways and spend half and hour figuring out what broke-> get asked to change something -> have fun modifying stuff in my development environment -> …
Is it always like this or am I missing something?
I am just a python/django enthusiast. I know about css and html, but I am not an engineer by any means.
I really enjoy developing in Django but why is deployment hell?
/r/django
https://redd.it/1gwly9h
HPC-Style Job Scripts in the Cloud
The first parallel computing system I ever used were job scripts on HPC Job schedulers (like SLURM, PBS, SGE, ...). They had an API straight out of the 90s, but were super straightforward and helped me do research when I was still just a baby programmer.
The cloud is way more powerful than these systems, but kinda sucks from a UX perspective. I wanted to replicate the experience I had on HPC on the cloud with Cloud-based Job Arrays. It wasn't actually all that hard.
[Post here](https://docs.coiled.io/blog/slurm-job-arrays.html)
Video here
This is still super new (we haven't even put up proper docs yet) but I'm excited about the feature. Thoughts/questions/critiques welcome.
/r/Python
https://redd.it/1gwj7e6
Best Tech Stack for a Chat App with AI: Python vs Nest.js for Backend?
I am working on a B2C startup and need to design the backend for a website and mobile apps supporting a chat application. The platform will incorporate AI/ML models to analyze chats and user inputs, alongside a notification system for users. My initial idea is to separate the backend and AI services. Should I use Python for both the backend(with flask or django) and AI components, or would it be better to leverage Nest.js for the backend, while using Python for AI?
/r/flask
https://redd.it/1gwatcn
Generate Realistic Podcast Sessions Programmatically
Hey everyone! 👋
I just released `podcast_tts`, a Python library that generates **realistic podcasts and dialogues** with multi-speaker audio, background music, and professional-quality mixing—all running **100% locally**.
# What My Project Does
`podcast_tts` allows you to programmatically create high-quality audio sessions with multiple speakers, dynamic or premade voice profiles, and customizable background music. You can save the output as MP3 or WAV files and even assign playback to specific audio channels for spatial separation.
It’s designed to be flexible, whether you’re building an API with FastAPI or experimenting with personal projects.
# Target Audience
This library is perfect for:
* Developers needing a **local TTS solution** for privacy or offline use.
* Engineers building backend systems for **audio generation** (e.g., podcasts or virtual assistants).
* Anyone looking for an all-in-one tool for **dialogue generation** with professional audio quality.
# Comparison to Alternatives
Unlike many TTS libraries that rely on cloud services, `podcast_tts` is fully offline, ensuring privacy and reducing latency. It also integrates features like **multi-speaker support**, **background music mixing**, and **text normalization**, which are often missing or require multiple tools to achieve.
The project is **open source**, and you can find it on GitHub here: [GitHub Repo](https://github.com/puntorigen/podcast_tts).
It’s also available on **PyPI** for easy installation: `pip install podcast_tts`.
I’ve shared more
/r/Python
https://redd.it/1gw5j21
Django REST Framework (DRF) ?
I have a strong foundation in Django and have completed several full-stack projects using Django templates. Now that I’m confident with the basics, I’m looking to expand my skills by diving into Django REST Framework (DRF) and building APIs.
I already understand the core concepts of APIs and how they work, but I’m looking for high-quality resources to help me get started with DRF whether it’s books, video tutorials, or other learning materials.
If you have any recommendations, I’d greatly appreciate your guidance. Thank you!
/r/djangolearning
https://redd.it/1gvx77f
Offering 50 free places on my Python Udemy course
Thank you to everyone who beta tested my new Udemy course "The 10-Day Python Bootcamp for Engineers and Scientists".
Things are going well and I'm making some income! Given there are literally millions of people out there interested in Python, I figured it wouldn't do any harm to hand out some more vouchers to this community.
As such, here is a link to 50 free vouchers for the course: https://www.udemy.com/course/python-for-engineers-scientists-and-analysts/?couponCode=THANKYOUREDDIT
As always, I'm grateful for your feedback. Enjoy the course if you do take a voucher.
/r/Python
https://redd.it/1gw30h9
Just published part 2 of my articles on Python Project Management and Packaging, illustrated with uv
Hey everyone,
Just finished the second part of my comprehensive guide on Python project management. This part covers both building packages and publishing.
It's like the first article, the goal is to dig in the PEPs and specifications to understand what the standard is, why it came to be and how. This is was mostly covered in the build system section of the article.
The article: https://reinforcedknowledge.com/a-comprehensive-guide-to-python-project-management-and-packaging-concepts-illustrated-with-uv-part-2/
I have tried to implement some of your feedback. I worked a lot on the typos (I believe there aren't any but I may be wrong), and I tried to divide the article into three smaller articles:
- Just the high level overview: https://reinforcedknowledge.com/a-comprehensive-guide-to-python-project-management-and-packaging-part-2-high-level-overview/
- The deeper dive into the PEPs and specs for build systems: https://reinforcedknowledge.com/a-comprehensive-guide-to-python-project-management-and-packaging-part-2-source-trees-and-build-systems-interface/
- The deeper dive into PEPs and specs for package formats: https://reinforcedknowledge.com/a-comprehensive-guide-to-python-project-management-and-packaging-part-2-sdists-and-wheels/
In the parent article there are also two smalls sections about uv build and uv publish. I don't think they deserve to be in a separate smaller article and I included them for completeness but anyone can just go uv help <command> and read about the command and it'd be much better. I did explain some small details that I believe that not everyone knows but I don't think it replaces your
/r/Python
https://redd.it/1gw1fe6
[D] PhD in RL/ML Theory or LLM
Hi guys,
I'm at a crossroads in my academic journey and would appreciate the community's insights. I'm trying to decide between pursuing a PhD focused on reinforcement learning/ML theory versus specializing in large language models with more experimental/applied research (these are the only two offers I had).
# Key considerations are the following:
# Research Impact
* RL/ML Theory: Foundational work that could advance the field's mathematical understanding
* LLMs: Direct applications in today's most transformative AI systems
# Job Prospects
* Theory: Academia, research labs, potentially more limited industry roles
* LLMs: High industry demand, active research area in both academia and industry
# Long-term Relevance
* Theory: Core principles likely to remain valuable regardless of specific technologies
* LLMs: Currently revolutionary but uncertain long-term trajectory
Personal background
* I'm an international student and about to finish my master program in US, so I no longer has enough time before making the final decision. I used to research in ml theory, but did not end up with a real top conference publication in theory. I personally doubt if I have enough mathematical background to pursue a successful PhD in this area (e.g., at least publish 2 theory papers a year on ICML/NeurIPS/ICLR/COLT/AISTATS). At the same time, I am personally doubting if theory
/r/MachineLearning
https://redd.it/1gvx8vx
Multiple domains and allowedhosts
I built a landing page app with different domain for each page, stored in a model with domain as a field. Is there a way to dynamically create ALLOWED\HOSTS in settings? I have tried this and it doesn't work because models have loaded yet. I want to avoid having a giant list of domains in my settings file and redeploying every time that needs to change. I would rather add LandingPages in the django admin and have it update allowed hosts automatically.
ALLOWEDHOSTS = list(LandingPage.objects.valueslist('domain', flat=True))
/r/django
https://redd.it/1gvth59
moka-py: A high performance caching library for Python written in Rust with TTL/TTI support
Hello!
I'm exited to share my first Rust lib for Python — [moka-py](https://github.com/deliro/moka-py)!
# What My Project Does
**moka-py** is a Python binding for the highly efficient [Moka](https://github.com/moka-rs/moka) caching library written in Rust. This library allows you to leverage the power of Moka's high-performance, feature-rich cache in your Python projects.
# Key Features:
* **Synchronous Cache:** Supports thread-safe, in-memory caching for Python applications.
* **TTL Support:** Automatically evicts entries after a configurable time-to-live (TTL).
* **TTI Support:** Automatically evicts entries after a configurable time-to-idle (TTI).
* **Size-based Eviction:** Automatically removes items when the cache exceeds its size limit using the TinyLFU policy.
* **Concurrency:** Optimized for high-performance, concurrent access in multi-threaded environments.
* **Fully typed:** mypy/pyright friendly. Even decorators
# Example (`@lru_cache` drop-in replacement but with TTL and TTI support):
```
from time import sleep
from moka_py import cached
@cached(maxsize=1024, ttl=10.0, tti=1.0)
def f(x, y):
print("hard computations")
return x + y
f(1, 2) # calls computations
f(1, 2) # gets from the cache
sleep(1.1)
f(1, 2) # calls computations (since TTI has passed)
```
# One more example:
```
from time import sleep
from moka_py import Moka
# Create a cache with a capacity of 100 entries, with a TTL of 30 seconds
# and a TTI of 5.2 seconds. Entries are always removed after 30 seconds
# and are removed after 5.2 seconds if there are
/r/Python
https://redd.it/1gvnsoh
Django Rest Framework Multiple Instance Creation tip
/r/django
https://redd.it/1gvnosg
Created an AI Research Assistant that actually DOES research! one query FULL document of knowledge!
Automated-AI-Web-Researcher: After months of work, I've made a python program that turns local LLMs running on Ollama into online researchers for you, Literally type a single question or topic and wait until you come back to a text document full of research content with links to the sources and a summary and ask it questions too! and more!
What My Project Does:
This automated researcher uses internet searching and web scraping to gather information, based on your topic or question of choice, it will generate focus areas relating to your topic designed to explore various aspects of your topic and investigate various related aspects of your topic or question to retrieve relevant information through online research to respond to your topic or question. The LLM breaks down your query into up to 5 specific research focuses, prioritising them based on relevance, then systematically investigates each one through targeted web searches and content analysis starting with the most relevant.
Then after gathering the content from those searching and exhausting all of the focus areas, it will then review the content and use the information within to generate new focus areas, and in the past it has often finding new, relevant focus areas based on
/r/Python
https://redd.it/1gvlzax
I'm stucking in installing numpy
Hello everyone , im trying to install numpy library on my terminal and always i face a problem on installing backend dependencies, i try to solve this problem by adding cmake to my system path and the c/cpp compiler and the problem continue , then i try installing .whl files of numpy and still the same problem , can anyone here have a solution for this problem .
/r/Python
https://redd.it/1gvkpbm
Wednesday Daily Thread: Beginner questions
# Weekly Thread: Beginner Questions 🐍
Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you.
## How it Works:
1. Ask Anything: Feel free to ask any Python-related question. There are no bad questions here!
2. Community Support: Get answers and advice from the community.
3. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources.
## Guidelines:
This thread is specifically for beginner questions. For more advanced queries, check out our [Advanced Questions Thread](#advanced-questions-thread-link).
## Recommended Resources:
If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.
## Example Questions:
1. What is the difference between a list and a tuple?
2. How do I read a CSV file in Python?
3. What are Python decorators and how do I use them?
4. How do I install a Python package using pip?
5. What is a virtual environment and why should I use one?
Let's help each other learn Python! 🌟
/r/Python
https://redd.it/1gvbziu