1234
Hot data science related posts every hour. Chat: https://telegram.me/r_channels Contacts: @lgyanf
[D] Attention before it was all we needed
*hey all,*
so I guess most of us have read/heard of *Attention Is All You Need*, which gave us the foundation of the transformer models we all use today. Yesterday I spent some time browsing some pre-cursor papers that were exploring attention right before the AIAYN paper. The ones I found most relevant were:
* End-To-End Memory Networks: [https://arxiv.org/pdf/1503.08895](https://arxiv.org/pdf/1503.08895)
* Key-Value Memory Networks for Directly Reading Documents: [https://arxiv.org/pdf/1606.03126](https://arxiv.org/pdf/1606.03126)
* Neural Machine Translation by Jointly Learning to Align and Translate: [https://arxiv.org/pdf/1409.0473](https://arxiv.org/pdf/1409.0473)
they all (directly or indirectly) use something like the `softmax(QK^T)V` (scaled dot-product attention, SDPA) operation in different ways, but with extra machinery on top, which makes them feel less general and more specialized to a particular setup.
it’s kind of fun in hindsight that this core calculation was almost a “trick” in these earlier works, embedded into more complex systems, and then AIAYN comes along and says: actually, let’s strip away most of the extra parts and just make attention the main building block — “attention is all you need”.
Hope some of you find this interesting. I’d love to hear any insights or anecdotes from people who were around / working with these models at the time. and if there are other important pre-transformer attention papers I should read, please let me know as well. ⚡
/r/deeplearning
https://redd.it/1pc37u0
AMA with Indiana University CL Faculty on November 24
Hi r/LanguageTechnology! Three of us faculty members here in [computational linguistics at Indiana University Bloomington](https://cl.indiana.edu/) will be doing an AMA on this coming Monday, **November 24**, from **2pm to 5pm ET** (19 GMT to 22 GMT).
The three of us who will be around are:
* [Luke Gessler](https://lgessler.com/) (low-resource NLP, corpora, computational language documentation)
* [Shuju Shi](https://scholar.google.com/citations?user=SGZk95cAAAAJ&hl=en) (speech recognition, phonetics, computer-aided language learning)
* [Sandra Kuebler](https://cl.indiana.edu/~skuebler/) (parsing, hate speech, machine learning for NLP)
We're happy to field your questions on:
* Higher education in CL
* MS and PhD programs
* Our research specialties
* Anything else on your mind
Please save the date, and look out for the AMA thread which we'll make earlier in the day on the 24th.
EDIT: we're going to reuse this thread for questions, so ask away!
/r/LanguageTechnology
https://redd.it/1p263p0
Theory for Karpathy's "Zero to Hero"
I always enjoyed "understanding" how LLMs work but never actually implemented it. After a friend recommended "zero to hero", I have been hooked!!
I am just 1.5 videos in, but still feel there are gaps in what I am learning. I am also implementing the code myself along with watching.
I took an ML class in my college but its been 8 years and I don't remember much.
He mentions some topics like "cross entropy loss", "learning rate decay" or "maximum likelihood estimation", but don't necessarily go in depth. I want to structure my learnings more.
Can someone please suggest reading material to read along with these videos or some pre-requisites? I do not want to fall in tutorial trap.
/r/deeplearning
https://redd.it/1p2lm6z
I think we found a third phase of grokking — has anyone else seen this?
/r/deeplearning
https://redd.it/1oyrmfy
Linguistics Student looking for career advice
I'm currently in my third year of my Linguistics degree. Next year (2026-2027) will be my last and I will specialize in Computational Linguistics. I would like to get into the world of NLP Engineering, or NLP in any way. What can I do courses or certificates wise? I would like to start working asap, and I wouldn't mind doing a Master's degree while I work. Any recommendation or suggestion is welcome 😁
/r/LanguageTechnology
https://redd.it/1oqshsp
Wheres the Best Place to Rent a GPU for Model Training
Im planning some AI model training and want to rent a powerful GPU like an RTX 4090 instead of buying onejust curious.
Which platforms do you usually use Hows the pricing and availability in your area ?
/r/deeplearning
https://redd.it/1onzsyi
Google PhD Fellowship recipients 2025 D
Google have just announced the 2025 recipients.
What are the criteria to get this fellowship?
https://research.google/programs-and-events/phd-fellowship/recipients/
/r/MachineLearning
https://redd.it/1ogy6z9
Dual 3D vision | software/library - synced TEMAS modules
/r/computervision
https://redd.it/1o8uyy4
Face Reidentification Project 👤🔍🆔
/r/computervision
https://redd.it/1o0e88e
basketball players recognition with RF-DETR, SAM2, SigLIP and ResNet
/r/computervision
https://redd.it/1nv4d8u
R ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
We released ShinkaEvolve, a new state-of-the-art and fully open-source framework for program optimization, which we specifically designed to be easily integrated into any scientific codebase.
Open source code: https://github.com/SakanaAI/ShinkaEvolve
Technical report: https://arxiv.org/abs/2509.19349
Blog: https://sakana.ai/shinka-evolve/
You can start playing with ShinkaEvolve without even downloading any code, all inside a remote Google Colab instance: https://colab.research.google.com/github/SakanaAI/ShinkaEvolve/blob/main/examples/shinka\_tutorial.ipynb
In our technical report, we show how ShinkaEvolve can be easily applied across different problem domains. On the canonical circle packing task, ShinkaEvolve discovers a new solution with state-of-the-art performance beyond the recent closed-source AlphaEvolve using only 150 program evaluations. We even apply ShinkaEvolve to small-scale LLM pretraining, discovering a new load-balancing loss for MoE architectures with remarkable stabilization properties.
ShinkaEvolve also comes with a detailed and lightweight WebUI to monitor its discoveries in real-time!
/r/MachineLearning
https://redd.it/1nq856v
R NeurIPS rejected paper resubmission
My paper just got rejected (scores: 4, 4, 3, 3). I’m considering resubmitting it to IEEE SatML. What’s your opinion on SatML? Would it be better to aim for a journal like IEEE TIFS instead? Any other recommendations? I’m not really interested in ICLR since I feel it might get rejected there too. Field: AI Security.
/r/MachineLearning
https://redd.it/1nkrmzr
D Self-Promotion Thread
Please post your personal projects, startups, product placements, collaboration needs, blogs etc.
Please mention the payment and pricing requirements for products and services.
Please do not post link shorteners, link aggregator websites , or auto-subscribe links.
\--
Any abuse of trust will lead to bans.
Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
\--
Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.
/r/MachineLearning
https://redd.it/1n67lft
M4 Mac Mini for real time inference
Nvidia Jetson nanos are 4X costlier than they are in the United States so I was thinking of dealing with some edge deployments using a M4 mini mac which is 50% cheaper with double the VRAM and all the plug and play benefits, though lacking the NVIDIA accelerator ecosystem.
I use a M1 Air for development (with heavier work happening in cloud notebooks) and can run RFDETR Small at 8fps atits native resolution of 512x512 on my laptop. This was fairly unoptimized
I was wondering if anyone has had the chance of running it or any other YOLO or Detection Transformer model on an M4 Mini Mac and experienced a better performance -- 40-50fps would be totally worth it overall.
Also, my current setup just included calling the model.predict function, what is the way ahead for optimized MPS deployments? Do I convert my model to mlx? Will that give me a performance boost? A lazy question I admit, but I will be reporting the outcomes in comments later when I try it out after affirmations.
Thank you for your attention.
/r/computervision
https://redd.it/1n563nj
I built a tool to benchmark tokenizers across 100+ languages and found some wild disparities R
TL;DR: Created tokka-bench to compare tokenizers across languages. Turns out your fine-tune's multilingual performance might suck because of tokenization, not architecture. Also explains why proprietary models (Claude, GPT, Gemini) are so much better at non-English tasks.
Links:
[Live dashboard](https://tokka-bench.streamlit.app/)
Full blog post
[GitHub repo](https://github.com/bgub/tokka-bench)
https://preview.redd.it/7i03jela9elf1.png?width=1724&format=png&auto=webp&s=95378457970e6337b147e71d7a8f0ab2dd67cb91
# The Problem Nobody Talks About
I started this as a side quest while pretraining a multilingual model, but tokenization turned out to be way more important than expected. There are two hidden layers creating massive efficiency gaps:
UTF-8 encoding differences:
English: \~1 byte per character
Arabic: 2+ bytes per character
Chinese: 3+ bytes per character
Tokenization bias: Most tokenizers are trained on English-heavy data, so they allocate way more vocabulary to English patterns. These compound into serious problems.
# Why This Affects Performance
During training: If you allocate tokens proportionally (10M English, 1M Khmer), the Khmer text has WAY less semantic content because it needs more tokens per word. Plus Khmer tokens end up being character-level instead of semantic units, making concept storage much harder.
During inference: Low-resource languages need 2-3x more tokens per sentence:
Slower throughput (costs more to serve)
Context windows fill up faster
More chances to mess up during generation
# What I Built
tokka-bench measures four key things:
1. Efficiency \- bytes per token (compression quality)
2. Coverage \- unique tokens used (script representation)
3. Word splitting \- how often semantic units get fragmented
4. Subword fertility \- average tokens per semantic unit
# Interesting Findings
You can actually reverse-engineer training data from tokenizer performance:
Kimi K2: Exceptional Mandarin coverage (obviously Chinese-trained)
Gemma 3: Strong Urdu/Hindi performance
gpt-oss: Good Arabic/Gujarati coverage
Weirdest finding: Programming languages show almost identical efficiency across all tokenizers. Probably because everyone trains on GitHub with similar language distributions.
# Technical Details
Built on high-quality datasets (FineWeb, FineWeb-2, StarCoder). Samples 2MB per language and calculates per-language metrics. Has some limitations around cross-linguistic comparison due to UTF-8 differences, but great for comparing tokenizers on the same language.
Shoutout to Judit Ács for the original subword fertility metrics and Rust et al's ACL paper that laid the groundwork.
PS: if you're from an AI lab and want to contribute your tokenizer's metrics (even if proprietary), please reach out! The community would benefit a lot from understanding how SOTA systems handle this stuff.
Posted this on LinkedIn/Twitter already but figured r/MachineLearning would appreciate the technical details. Happy to answer questions about methodology or findings!
/r/MachineLearning
https://redd.it/1n0r8b7
Did self-supervised learning for visual features quietly peak already?
From around 2020–2024 it felt like self-supervised learning (SSL, self-supervised learning) for image features was on fire — BYOL (Bootstrap Your Own Latent), SimCLR (Simple Contrastive Learning of Representations), SwAV (Swapping Assignments between multiple Views), DINO, etc. Every few months there was some new objective, augmentation trick, or architectural tweak that actually moved the needle for feature extractors.
This year it feels a lot quieter on the “new SSL objective for vision backbones” front. We got DINOv3, but as far as I can tell it’s mostly smart but incremental tweaks plus a lot of scaling in terms of data and compute, rather than a totally new idea about how to learn general-purpose image features.
So I’m wondering:
Have I just missed some important recent SSL image models for feature extraction?
Or has the research focus mostly shifted to multimodal/foundation models and generative stuff, with “vanilla” visual SSL kind of considered a solved or mature problem now?
is the SSL scene for general vision features still evolving in interesting ways, or did we mostly hit diminishing returns after the original DINO/BYOL/SimCLR wave?
/r/computervision
https://redd.it/1pavb20
Kimi K2 Thinking and Gemini 3 may have just shown OpenAI to be the AI bubble epicenter.
In an interview recently. Sam Altman commented that while he didn't think there was an AI bubble, some players were poised to lose a whole lot of money. Before Moonshot AI launched Kimi K2 Thinking on November 6 and before Google launched Gemini 3 on November 18, coming out of nowhere to massively leapfrog over every other AI by an historic margin, we might have wondered who these big losers in the AI race would ultimately be. Now that the numbers are in, it seems Altman might have presciently been talking about OpenAI.
Here's why. Let's begin with OpenAI's revenue projections for the next 5 years, all calculated before the launch of Kimi K2 Thinking and Gemini 3. A few key points stand out. First, OpenAI made those earnings projections about products that don't yet exist. Second, no one has yet created the demand for these products. And third, perhaps most importantly, OpenAI apparently didn't factor in the competition.
So when a 2-year-old startup from China open sources a thinking model it trained on less than $5 million, (by comparison GPT-5 cost OpenAI between $1.5 billion and $2 billion to train) you have to appreciate how much the AI landscape has shifted in a matter of days. And K2 Thinking was not just another model. It outperformed GPT-5. Grok 4, Gemini 2.5, and Claude 4 on many of the most important benchmarks. Of course the threat that OpenAI faces isn't really about Moonshot or Kimi K2 Thinking. It's about the world now knowing with absolute certainty that a small lab spending a miniscule amount of money can overtake ALL of the AI giants, while costing consumers and enterprises from 2 to 10 times less to run.
But Kimi K2 Thinking really isn't what OpenAI should be worried about. Let the following sink in:
Gemini 3 set monstrous new highs with 37.5% on Humanity’s Last Exam and 45.1% on ARC-AGI-2 in Deep Think mode—nearly doubling GPT-5 on both measures. It also scored 1501 Elo on LMArena and 91.9% on GPQA Diamond, outperforming GPT-5 and Claude across strategic reasoning, scientific knowledge, and abstract problem-solving. And that's just the beginning. Gemini 3 dominated its competitors far beyond those key benchmarks. If you're brave enough to review a brutally detailed account of how completely Gemini 3 trounced OpenAI and pretty much everyone else on pretty much everything, check out the following stats:
https://www.vellum.ai/blog/google-gemini-3-benchmarks?utm=&utmsource=direct&utmmedium=none
These scores position Gemini 3 way ahead -- perhaps years ahead -- of OpenAI on the metrics that matter most to both consumer and enterprise AI. Essentially Google just ate OpenAI's lunch, dinner and breakfast the next day.
But that's just the competition part of all of this. While Kimi K2 Thinking clearly demonstrates that massive data centers are just not necessary to building the most powerful AIs, OpenAI has committed $1.4 trillion in investments to build massive data centers, most of which won't be operational for years. It could be that this miscalculation -- this massive misappropriation of investment commitments -- best comes to explain why OpenAI may have positioned itself to be THE big loser in the AI bubble that Altman warned everyone about.
The bottom line is that if OpenAI doesn't pull a rabbit out of the hat during 2026, it may become the first major casualty of the AI bubble that will hopefully be limited to colossally unwise investments like those of OpenAI. For their sake, let's hope that it's a really, really big rabbit.
/r/deeplearning
https://redd.it/1p558ag
Apple AIML Residency Program 2026 R
Haven't seen a 2026 post - wanted to use this to consolidate info from everyone on the process. Anyone have any idea when they start sending out info session updates?
/r/MachineLearning
https://redd.it/1p0lart
D Monthly Who's Hiring and Who wants to be Hired?
For Job Postings please use this template
>Hiring: [Location\], Salary:[\], [Remote | Relocation\], [Full Time | Contract | Part Time\] and [Brief overview, what you're looking for\]
For Those looking for jobs please use this template
>Want to be Hired: [Location\], Salary Expectation:[\], [Remote | Relocation\], [Full Time | Contract | Part Time\] Resume: [Link to resume\] and [Brief overview, what you're looking for\]
​
Please remember that this community is geared towards those with experience.
/r/MachineLearning
https://redd.it/1okj2rw
How does Qwen3-Next Perform in Complex Code Generation & Software Architecture?
https://redd.it/1opwl9h
@datascientology
masters in computational linguistics uppsala or tübingen
hi all
i'm planning to apply for a masters in computational linguistics / language technology as an international (non EU/EEA) student. i've done research on programs and have narrowed down on these few:
1. uppsala's MA language technology masters
2. tübingen's MA computational linguistics
3. stockholm's MA AI and language
4. stuttgart's MSc Computational Linguistics
5. konstanz's MA speech and language processing
6. helsinki's MA linguistic diversity and digital humanities (language technology track)
7. potsdam's MSc cognitive systems
coming from a linguistic background (bachelor with honours), i'm looking at 2 year programs as i believe i'd be able to learn more programming theory + technical skills that would better equip me for an industry role in the tech sector. i'm thus not as keen on 1 year programs such as leiden's linguistics (comp ling track), VU's linguistics language and AI, or groningen's speech technology programs. i'm learning python online to gain some basic proficiency in programming before starting the masters.
uppsala and tübingen are my top 2 choices if i were to be accepted, particularly because they seem more accessible to prospective students from a linguistic background based on my research. i'm hoping to gain more information about these two cities and their programs based on people's personal experience so that i can make an informed choice. these are my questions:
1. ACCESSIBILITY: how accessible is the program for those with a linguistic background? accessible could mean being less CS-intensive, or that there are foundational classes in programming/ML/AI to help those with a humanities background ease into the program with less difficulty
2. TEACHING QUALITY: what's your experience with the quality of teaching, how well organised the course is, helpfulness of professors, whether studying resources are provided or you'd have to source for your own materials, etc
3. JOB OPPORTUNITIES: in which city would an international student find it easier to get a job after graduating?
4. HEALTHCARE: how easy is it to get a medical appointment for minor and major illnesses in the city, both as a student and after graduation?
5. SOCIAL LIFE: how open people are to making new (local) friends, especially if one is not fluent in Swedish (for uppsala) or German (for tübingen)?
6. ACTIVITIES: which city has more options for activities if i'm not a huge fan of partying, alcohol, pub crawls? (occasional outings for special occassions are fine, but it's not something i would do frequently or particularly enjoy) i'm open to hiking, bouldering, music events, board games, reading, or any other activity
7. TRANSPORT: how well-connected and accessible is public transport within these cities, and also from the city to other cities?
8. COST OF LIVING: it seems like living costs (on numbeo) are generally lower in uppsala than tübingen (which is counter to my initial impression that CoL is higher in nordic countries) and i'm wondering if this is really the case? i've also read comments that tübingen is an expensive city to live in - would this make the cost of living in tübingen 'comparable' to uppsala?
9. QUALTITY OF LIFE: how would you describe the overall quality of life in uppsala/tübingen, and if you have experience living in both, is the quality of life noticeably better in one of the cities? (my impression is that anywhere in the nordics would have a better quality of life but i'd like to hear your experience if you've lived there)
i'd be grateful if you could share your experience in uppsala and/or tübingen, or if you have experience with the other programs (and countries). thanks so much!
TLDR: international student (non EU/EEA) with BA (Honours) in Linguistics looking for advice on whether to choose uppsala or tübingen for masters in computational linguistics/language technology
/r/LanguageTechnology
https://redd.it/1omgy7r
How was this achieved? They are able to track movements and complete steps automatically
/r/computervision
https://redd.it/1oeosd3
D Self-Promotion Thread
Please post your personal projects, startups, product placements, collaboration needs, blogs etc.
Please mention the payment and pricing requirements for products and services.
Please do not post link shorteners, link aggregator websites , or auto-subscribe links.
\--
Any abuse of trust will lead to bans.
Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
\--
Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.
/r/MachineLearning
https://redd.it/1nvrmw5
D Monthly Who's Hiring and Who wants to be Hired?
For Job Postings please use this template
>Hiring: [Location\], Salary:[\], [Remote | Relocation\], [Full Time | Contract | Part Time\] and [Brief overview, what you're looking for\]
For Those looking for jobs please use this template
>Want to be Hired: [Location\], Salary Expectation:[\], [Remote | Relocation\], [Full Time | Contract | Part Time\] Resume: [Link to resume\] and [Brief overview, what you're looking for\]
​
Please remember that this community is geared towards those with experience.
/r/MachineLearning
https://redd.it/1nuwj5t
I built TagiFLY – a lightweight open-source labeling tool for computer vision (feedback welcome!)
Hi everyone,
Most annotation tools I’ve used felt too heavy or cluttered for small projects. So I created TagiFLY – a lightweight, open-source labeling app focused only on what you need.
🔹 What it does
6 annotation tools (box, polygon, point, line, mask, keypoints)
4 export formats: JSON, YOLO, COCO, Pascal VOC
Light & dark theme, keyboard shortcuts, multiple image formats (JPG, PNG)
🔹 Why I built it
I wanted a simple tool to create datasets for:
🤖 Training data for ML
🎯 Computer vision projects
📊 Research or personal experiments
https://preview.redd.it/y28zj1mftyrf1.png?width=1748&format=png&auto=webp&s=6aeb618cdfd4276c07c1e94c79763a4074ff8334
Export Window
🔹 Demo & Code
👉 GitHub repo: https://github.com/dvtlab/tagiFLY
⚠️ It’s still in beta – so it may have bugs or missing features.
I’d love to hear your thoughts:
Which features do you think are most useful?
What would you like to see added in future versions?
Thanks a lot 🚀
/r/computervision
https://redd.it/1nsyv0h
D Is non-DL related research a poor fit for ICLR?
I was one of the lucky people rejected from NEURIPS with 6444 scores but cranky AC, so looking to resubmit now. Since it got good reviews at NEURIPS, I'm considering submitting to ICLR incorporating suggested changes.
However, my paper proposes a linear dimensionality reduction technique, based on information geometry. It is my understanding that ICLR is very focused on neural networks and Deep Learning, so I am worried that my paper is not a good fit, so also considering AISTATS.
Is a novel linear dimensionality reduction technique too out of scope for ICLR? I am an outsider to the field, so would very much appreciate opinions.
/r/MachineLearning
https://redd.it/1nn56yu
D Monthly Who's Hiring and Who wants to be Hired?
For Job Postings please use this template
>Hiring: [Location\], Salary:[\], [Remote | Relocation\], [Full Time | Contract | Part Time\] and [Brief overview, what you're looking for\]
For Those looking for jobs please use this template
>Want to be Hired: [Location\], Salary Expectation:[\], [Remote | Relocation\], [Full Time | Contract | Part Time\] Resume: [Link to resume\] and [Brief overview, what you're looking for\]
​
Please remember that this community is geared towards those with experience.
/r/MachineLearning
https://redd.it/1n4jdo7
Tried building an explainable Vision-Language Model with CLIP to spot and explain product defects!
/r/deeplearning
https://redd.it/1n6lpte
AI research is drowning in papers that can’t be reproduced. What’s your biggest reproducibility challenge?
Curious — what’s been your hardest challenge recently? Sharing your own outputs, reusing others’ work?
We’re exploring new tools to make reproducibility proofs verifiable and permanent (with web3 tools, i.e. ipfs), and would love to hear your inputs.
The post sounds a little formal, as we are reaching a bunch of different subreddits, but please share your experiences if you have any, I’d love to hear your perspective.
Mods, if I'm breaking some rules, I apologize, I read the subreddit rules, and I didn't see any clear violations, but if I am, delete my post and don't ban me please :c.
/r/LanguageTechnology
https://redd.it/1mzxfkz
The best tools I’ve found for evaluating AI voice agents
I’ve been working on a voice agent project recently and quickly realized that building the pipeline (STT → LLM → TTS) is the easy part. The real challenge is evaluation, making sure the system performs reliably across accents, contexts, and multi-turn conversations.
I went down the rabbit hole of voice eval tools and here are the ones I found most useful:
1. **Deepgram Eval**
* Strong for transcription accuracy testing.
* Provides detailed WER (word error rate) metrics and error breakdowns.
2. **Speechmatics**
* I used this mainly for multilingual evaluation.
* Handles accents/dialects better than most engines I tested.
3. **Voiceflow Testing**
* Focused on evaluating conversation flows end-to-end.
* Helpful when testing dialogue design beyond just turn-level accuracy.
4. **Play.ht Voice QA**
* More on the TTS side, quality and naturalness of synthetic voices.
* Useful if you care about voice fidelity as much as the NLP part.
5. **Maxim AI**
* This stood out because it let me run *structured evals on the whole voice pipeline*.
* Latency checks, persona-based stress tests, and pre/post-release evaluation of agents.
* Felt much closer to “real user” testing than just measuring WER.
I’d love to hear if anyone here has explored other approaches to **systematic evaluation of voice agents,** especially for multi-turn robustness or human-likeness metrics.
/r/LanguageTechnology
https://redd.it/1mufzbv
