1102
Daily Python News Question, Tips and Tricks, Best Practices on Python Programming Language Find more reddit channels over at @r_channels
Nginx 404'ing all images.
I'm not sure if this should be in the nginx or Django Reddit, but I'll try here first. My blog is running on Docker. Initially, all images in the static files from the first set of articles I created while coding the blog were accessible to nginx. However, when I tried adding articles from the admin panel after deployment, the new images returned a 404 error. I tried debugging by checking my code and realized I didn't include a path for the media folder in the `settings.py` file. After adding that line and rebuilding the container... well, now the previously accessible images are returning a 404. I think my nginx server might not be configured correctly. *I've entered the container and verified that files are present*
Dockerfile:
# Use the official Python image from the Docker Hub
FROM python:3.11
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
# Set the working directory
WORKDIR /app
# Copy the requirements file into the container
COPY requirements.txt /app/
# Install the dependencies
RUN pip install --upgrade pip && pip install -r requirements.txt
# Copy the entire project into the container
COPY . /app/
# Collect static files
RUN python manage.py collectstatic --noinput
EXPOSE 1617
# Run the Gunicorn server
CMD ["gunicorn", "redacted.wsgi:application", "--bind", "0.0.0.0:1617"\]
docker-compose:
version: '3'
services:
web:
build: .
command: gunicorn --workers
/r/django
https://redd.it/1g0cvxr
[blog post] Hugging Face + Dask for parallel data processing and model inference
Wanted to share a blog post on using Hugging Face with Dask to process the FineWeb dataset. The example goes through:
* Reading directly from Hugging Face with Dask, eg `df = dask.dataframe.read_parquet(hf://...)`
* Using a Hugging Face Language Model to classify the educational level of the text.
* Filtering the highly educational web pages as a new dataset and writing in parallel directly from Dask to Hugging Face storage.
The example goes through processing a small subset of the FineWeb dataset with pandas and then scaling out to multiple GPUs with Dask.
Blog post: [https://huggingface.co/blog/dask-scaling](https://huggingface.co/blog/dask-scaling)
/r/Python
https://redd.it/1fzyh7x
ParScrape v0.4.6 Released
# What My project Does:
Scrapes data from sites and uses AI to extract structured data from it.
# Whats New:
Added more AI providers
Updated provider pricing data
Minor code cleanup and bug fixes
Better cleaning of HTML
# Key Features:
Uses Playwright / Selenium to bypass most simple bot checks.
Uses AI to extract data from a page and save it various formats such as CSV, XLSX, JSON, Markdown.
Has rich console output to display data right in your terminal.
# GitHub and PyPI
PAR Scrape is under active development and getting new features all the time.
Check out the project on GitHub or for full documentation, installation instructions, and to contribute: [https://github.com/paulrobello/par\_scrape](https://github.com/paulrobello/par_scrape)
PyPI https://pypi.org/project/par\_scrape/
# Comparison:
I have seem many command line and web applications for scraping but none that are as simple, flexible and fast as ParScrape
# Target Audience
AI enthusiasts and data hungry hobbyist
/r/Python
https://redd.it/1g06arb
Thursday Daily Thread: Python Careers, Courses, and Furthering Education!
# Weekly Thread: Professional Use, Jobs, and Education 🏢
Welcome to this week's discussion on Python in the professional world! This is your spot to talk about job hunting, career growth, and educational resources in Python. Please note, this thread is not for recruitment.
---
## How it Works:
1. Career Talk: Discuss using Python in your job, or the job market for Python roles.
2. Education Q&A: Ask or answer questions about Python courses, certifications, and educational resources.
3. Workplace Chat: Share your experiences, challenges, or success stories about using Python professionally.
---
## Guidelines:
- This thread is not for recruitment. For job postings, please see r/PythonJobs or the recruitment thread in the sidebar.
- Keep discussions relevant to Python in the professional and educational context.
---
## Example Topics:
1. Career Paths: What kinds of roles are out there for Python developers?
2. Certifications: Are Python certifications worth it?
3. Course Recommendations: Any good advanced Python courses to recommend?
4. Workplace Tools: What Python libraries are indispensable in your professional work?
5. Interview Tips: What types of Python questions are commonly asked in interviews?
---
Let's help each other grow in our careers and education. Happy discussing! 🌟
/r/Python
https://redd.it/1g05xw8
in 2024 learn flask or django?
hi everyone, i was wonder which one of these frameworks is better and worth to learn and make money? flask? django? or learn both?
/r/flask
https://redd.it/1fzsrct
Speeding up unit tests in CI/CD
I have a large Django project that currently takes ca. 30 minutes to run all the unit tests serially in our CI/CD pipeline and we want speed this up as it's blocking our releases.
I have a Ruby background and am new to Python - so I'm investigating the options available in the Python ecosystem to speed this up. So far I've found:
[pytest-xdist](https://pypi.org/project/pytest-xdist/)
pytest-split
[pytest-parallel](https://pypi.org/project/pytest-parallel/)
pytest-run-parallel
[tox](https://tox.wiki/en/latest/index.html) parallel (not exactly what I need, as I only have one environment)
CircleCI's test splitting - I've used this for Ruby, and it didn't do so well when some classes had a lot of tests in them
I'd love to hear your experiences of these tools and if you have any other suggestions.
/r/Python
https://redd.it/1fzreee
N Jurgen Schmidhuber on 2024 Physics Nobel Prize
The NobelPrizeinPhysics2024 for Hopfield & Hinton rewards plagiarism and incorrect attribution in computer science. It's mostly about Amari's "Hopfield network" and the "Boltzmann Machine."
1. The Lenz-Ising recurrent architecture with neuron-like elements was published in 1925 . In 1972, Shun-Ichi Amari made it adaptive such that it could learn to associate input patterns with output patterns by changing its connection weights. However, Amari is only briefly cited in the "Scientific Background to the Nobel Prize in Physics 2024." Unfortunately, Amari's net was later called the "Hopfield network." Hopfield republished it 10 years later, without citing Amari, not even in later papers.
2. The related Boltzmann Machine paper by Ackley, Hinton, and Sejnowski (1985) was about learning internal representations in hidden units of neural networks (NNs) S20. It didn't cite the first working algorithm for deep learning of internal representations by Ivakhnenko & Lapa. It didn't cite Amari's separate work (1967-68) on learning internal representations in deep NNs end-to-end through stochastic gradient descent (SGD). Not even the later surveys by the authors nor the "Scientific Background to the Nobel Prize in Physics 2024" mention these origins of deep learning. (BM also did not cite relevant prior work by Sherrington & Kirkpatrick &
/r/MachineLearning
https://redd.it/1fzw5b1
Do multiple requests create multiple instances of a middleware object?
For example, let's say you have some middleware that does something with process_view(), to run some code before a view is rendered. If you have you Django app deployed with Guincorn and Nginix, does every client request get its own instantiation of that middleware class? Or do they all share the same instantiation of that object in memory wherever your code is deployed? (or does each of Gunicorn's workers create its own instantiation?)
/r/djangolearning
https://redd.it/1fzswzm
N The 2024 Nobel Prize in Chemistry goes to the people Google Deepmind's AlphaFold. One half to David Baker and the other half jointly to Demis Hassabis and John M. Jumper.
Announcement: https://twitter.com/NobelPrize/status/1843951197960777760
/r/MachineLearning
https://redd.it/1fznxyr
I Made Spotify Premium, But This Time It's Free
A while ago, I shared how I built a Python script to download my Spotify Liked Songs as MP3s. Since then, I decided to take the project one step further. Instead of manually running the script every time I added new songs, I’ve now automated the entire process to check for new liked songs every day and download them automatically.
Here’s how it works:
- Extracts my Liked Songs from Spotify using the spotipy library.
- Compares them with the list of songs already downloaded to avoid duplicates.
- Uses youtubesearchpython to find the top YouTube result for any new songs.
- Downloads the audio as an MP3 using yt-dlp and stores them in the Songs directory on my desktop.
It’s simple, but it now saves me even more time. I no longer have to worry about manually managing my favorite songs!
This project was so fun to work on, and now I've made it even smarter, by making it run automatically using the Windows built-in Task Scheduler!
I was thinking of making this like a paid tool, with a small cost of $2, but I figured let's share this with people so they can also benefit :)
Here's the GitHub repo:
https://github.com/aneeb02/SpotiFlopy
/r/Python
https://redd.it/1fzidjo
Why Django supports the Open Source Pledge
https://www.djangoproject.com/weblog/2024/oct/08/why-django-supports-the-open-source-pledge/
/r/django
https://redd.it/1fzipz8
Beginner's Guide for Django Deployment
Hey all,
I've noticed beginners struggling with Django deployment, so I wanted to share this free and open-source guide. It is beginner-friendly, explains the process clearly, and helps you get your project deployed quickly.
Any contributions are welcome from the community to improve this guide. If you find it useful, please consider giving the GitHub repo a star ⭐ (it helps a lot!)
Link: Beginner's Guide for Django Deployment
GitHub Repo: https://github.com/bhavya-tech/django-deployment
Feel free to ask questions here, I will be happy to help!
Happy coding!
/r/django
https://redd.it/1fz1t5d
I just wanna tell you guys that I did it!
I'm sorry oif this is not the right space but I didn't have someone to share this and would actually understand my happiness.
I was finally able to deploy my Django application on Digital Ocean with Postgres, Nginx, and Gunicorn on its Ubuntu machine. I also pointed my domain to the IP and set up the SSL certification. This took me days and I had to destroy the machine multiple times. Oh I also configured redis and celery with different workers.
I'm so hyped 😂😂
/r/django
https://redd.it/1fywr26
2024 Python Developers Survey
The 2024 Python Developers Survey run by the Python Software Foundation in partnership with JetBrains is now open!
Survey Link - 10-15 minutes to complete
Blog post
/r/Python
https://redd.it/1fz1siq
How to configure celery?
I am currently running a flask backend server that is used for various features and one of the features I have added is a model for prediction that takes about 5 to 10 minutes to train on hitting an api on frontend which works fine but the api itself timesout after 2 minutes although the model still gets trained and stored for each user. Now issue is to prevent a mad user experience I wanted to train the model in background and came across celery with redis which I was already for storing sessions so thought to use it and everything runs only just when I use @shared_task decorator to run my task it give Error 61 of connection refused although on running the celery command and backend it says celery is connected. Can someone help me here?
Edit - I used rabbitmq as the broker and redis as result backend and everything worked smoothly. Thanks everyone
/r/flask
https://redd.it/1fyajiz
Considering moving from Flask-Sqlalchemy to Flask and plain Sqlalchemy: not sure how to start, or if useful
Hi all,
I wrote a free language-learning tool called Lute. I'm happy with how the project's been going, I and a bunch of other people use it.
I wrote Lute using Flask, and overall it's been very good. Recently I've been wondering if I should have tried to avoid Flask-Sqlalchemy -- I was over my head when I started, and did the best I could.
My reasons for wondering:
- when I'm running some service or domain level tests, eg., I'm connecting to the db, but I'm not using Flask. It's just python code creating objects, calling methods, etc. The tests all need an app context to execute, but that doesn't seem like it's adding anything.
- simple data crunching scripts have to call the app initializer, and again push an app context, when really all I need is the service layer and domain objects. Unfortunately a lot of the code has stuff like "from lute.db import db" and "db.session.query() ...", etc, so the db usage is scattered around the code.
Today I hacked at changing it to plain sql alchemy, but it ended up spiralling out of my control, so I put that on ice to think a bit more.
These
/r/flask
https://redd.it/1g0ajo0
What to use instead of callbacks?
I have a lot of experience with Python, but I've also worked with JavaScript and Go and in some cases, it just makes sense to allow the caller to pass a callback (ore more likely a closure). For example to notify the caller of an event, or to allow it to make a decision. I'm considering this in the context of creating library code.
Python lambdas are limited, and writing named functions is clumsier than anonymous functions from other languages. Is there something - less clumsy, more Pythonic?
In my example, there's a long-ish multi-stage process, and I'd like to give the caller an opportunity to validate or modify the result of each step, in a simple way. I've considered class inheritance and mixins, but that seems like too much setup for just a callback. Is there some Python pattern I'm missing?
/r/Python
https://redd.it/1g02dtg
D Why is there so little statistical analyses in ML research?
Why is it so common in ML research to not do any statistical test to verify that the results are actually significant? Most of the times, a single outcome is presented, instead of doing multiple runs and performing something like a t-test or Mann Whitney U Test etc. Drawing conclusions based on a single sample would be impossible in other disciplines, like psychology or medicine, why is this not considered a problem in ML research?
Also, can someone recommend a book for exactly this, statistical tests in the context of ml?
/r/MachineLearning
https://redd.it/1fznaa9
Static files serving - S3 bucket alternative
Hello guys. I wanted to build an app with angular in frontend and Django in backend. The client must be able to click on a link and download a pdf. But the user must login to enter the app. How can I serve these pdfs? A friend told me about an S3 bucket. But is there any open source alternative for this? Is there any better solution? How to better integrate this solution with my Django authentication?
/r/django
https://redd.it/1fzuosv
How should I get started with Django?
I recently started to work with Django but I'm completely utterly humbled and devastated at the same time whenever I try to add a new function with an API call into my react project. I really don't understand the magic behind it and usually need to get help from other colleagues.
The Django documents are (I'm sorry) terrible. The more I read into it the more questions arise.
Are there any sources that can give me a better insight on how to work with API in Django and maybe API in general?
I appreciate any sources given (except Django docs)
/r/djangolearning
https://redd.it/1g02p1b
PEP 760 – No More Bare Excepts
PEP 760 – No More Bare Excepts
This PEP proposes disallowing bare except: clauses in Python’s exception-handling syntax.
- https://peps.python.org/pep-0760/
- https://discuss.python.org/t/pep-760-no-more-bare-excepts/
/r/Python
https://redd.it/1fzxwj3
What personal challenges have you solved using Python? Any interesting projects or automations?
Hey everyone! I'm curious—what have you used Python for in your daily life? Are there any small, repetitive tasks you've automated that made things easier or saved you time? I'd love to hear about it!
I stumbled upon an old article on this Python a while ago. I think it's worth revisiting this topic about it again.
/r/Python
https://redd.it/1fzupwm
is there a way to make pyright recognize related name fields?
from django.db import models
class A(models.Model):
id = models.IntegerField()
class B(models.Model):
id = models.IntegerField()
fkA = models.ForeignKey(A, ondelete=models.CASCADE, relatedname="fkB")
a = A(id=1)
b = B(id=2, fkA=a)
a.fkB #Here it says it cannot access attribute fkB for class A
take for example the code snippet above. is there a way to make pyright know that fk\B fields for class A exists?
/r/django
https://redd.it/1fzlr75
Routes not getting registered. 404 error
I'm using Flask for the first time, and I'm a baby programmer. This is my base template.
from flask import render_template
from app import app
from app.data_handler import DataHandler
data_handler = DataHandler()
@app.route('/')
def home():
return render_template('about.html')
@app.route('/about')
def about():
return render_template('about.html')
@app.route('/my_works')
def my_works():
projects = data_handler.get_projects()
print(projects) # Check what projects are loaded
return render_template('my_works.html', projects=projects)
@app.route('/project/<int:project_id>')
def project_detail(project_id):
project = data_handler.get_project(project_id)
if project is not None:
return render_template('project_detail.html', project=project)
else:
print("Project not found")
return "Project not found", 404
/r/flask
https://redd.it/1fzi7sd
so i made a download website LOL
Yeah, so I'll probably get tracked down by the government, but who cares? XD Look at that site!
Basically, it's a download site for movies. They're in zip files, etc.
FBI Comin for me boys
https://javu.xyz/
/r/flask
https://redd.it/1fzn2vq
Wednesday Daily Thread: Beginner questions
# Weekly Thread: Beginner Questions 🐍
Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you.
## How it Works:
1. Ask Anything: Feel free to ask any Python-related question. There are no bad questions here!
2. Community Support: Get answers and advice from the community.
3. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources.
## Guidelines:
This thread is specifically for beginner questions. For more advanced queries, check out our [Advanced Questions Thread](#advanced-questions-thread-link).
## Recommended Resources:
If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.
## Example Questions:
1. What is the difference between a list and a tuple?
2. How do I read a CSV file in Python?
3. What are Python decorators and how do I use them?
4. How do I install a Python package using pip?
5. What is a virtual environment and why should I use one?
Let's help each other learn Python! 🌟
/r/Python
https://redd.it/1fzeenv
Django bugfix release issued: 5.1.2
https://www.djangoproject.com/weblog/2024/oct/08/bugfix-releases/
/r/django
https://redd.it/1fz1xdr
I make a python library for use as wrapper and transform a pdf in html without lose format
Hello everyone, how are you? I recently created a library wrapper for PDF2HTMLEX for use in Python, this library can transform PDF files to HTML and also convert all PDF files in a folder to HTML semi-automatically.
I would appreciate your feedback by using it, making pull requests or stars if you like:
Repository URL:
https://github.com/gabriel-batistuta/pypdf2htmlEX
The library can be downloaded from PYPI as well:
https://pypi.org/project/pypdf2htmlex/
/r/Python
https://redd.it/1fz9ynu
Production level project help..
I am using boto3 with flask to convert video files (wth mediaConverter), after job done then only saving the video related data in mongodb, but how can I get to know the job is done, so I used sqs and SNS of AWS is it good in production level
Or u have some other approaches..
I want different scenarios and solutions instead of sns and sqs help with this please 🥺
/r/flask
https://redd.it/1fy7fi7
Loading private key of a certificate in local windows machine store
To use Microsoft graph API we use a service principal and a certificate. I can retrieve the certificate if it’s in azure key vault and use it to get an MSAL token for authentication. I am unable to do that if the certificate is in local windows machine certificate store.
Has anyone ever did this? I can’t find any working samples on the internet.
Would really appreciate a solution.
/r/Python
https://redd.it/1fz0ud0