Self-Compacting Language Model Agents
Paper β’ 2606.23525 β’ Published β’ 19
Ross Technologies
ross-dev
AI & ML interests
None yet
Recent Activity
new activity about 23 hours ago
Qwen/Qwen3.8-2.4T-A95B:Will this run on my phone? upvoted a paper about 2 months ago
Self-Compacting Language Model Agents published a bucket about 2 months ago
ross-dev/homeOrganizations
Will this run on my phone?
π§ π₯ 9
6
#10 opened 1 day ago
by 
Doctor-Chad-PhD
upvoted a paper about 2 months ago
published a
bucket about 2 months ago
Update config.json
2
#6 opened 7 months ago
by 
ChoLuckyHexSexySuccubusQueen135Age
published 3
datasets 6 months ago
updated a
Space 6 months ago
Request: DOI
1
#2 opened 7 months ago
by 
Chaiyphop
Looking for a cooperation.
2
#1 opened 10 months ago
by ross-dev
updated a
model 8 months ago
updated a
model 9 months ago
published 2
models 9 months ago
updated a
model 10 months ago
Update README.md
2
#5 opened 10 months ago
by 
mohammedmim
liked a
model 11 months ago
reacted to hba123's post with π 11 months ago
Post
2224
Hey, amazing, awesome people of the beautiful internet ππ₯°
Distillation has been (from my point of view) a main driving factor for the success of hashtag#LLMs - like distilling the knowledge of an amazing big model (say hashtag#DeepSeekv3, or hashtag#GeminiAI) into yours.
Probably, you have done it with minimising a KL divergence, and it somehow worked.
Well, not that well, right?
1οΈβ£ Your model tends to memorise!
2οΈβ£ Your model might get the right answer, but its reasoning might be flawed.
To fix those problems, we rethink distillation and process a new approach! A method that is based on constrained RL that comes with nice theoretical guarantees and excellent performance!
Check it out: Rethinking Large Language Model Distillation: A Constrained Markov Decision Process Perspective (2509.22921)
Let us do distillation right! Please upvote if you find it useful!
Distillation has been (from my point of view) a main driving factor for the success of hashtag#LLMs - like distilling the knowledge of an amazing big model (say hashtag#DeepSeekv3, or hashtag#GeminiAI) into yours.
Probably, you have done it with minimising a KL divergence, and it somehow worked.
Well, not that well, right?
1οΈβ£ Your model tends to memorise!
2οΈβ£ Your model might get the right answer, but its reasoning might be flawed.
To fix those problems, we rethink distillation and process a new approach! A method that is based on constrained RL that comes with nice theoretical guarantees and excellent performance!
Check it out: Rethinking Large Language Model Distillation: A Constrained Markov Decision Process Perspective (2509.22921)
Let us do distillation right! Please upvote if you find it useful!
Let's agree to AGREE!! :)
reacted to giadap's post with π 11 months ago
Post
11181
One of the hardest challenges in AI safety is finding the right balance: how do we protect people from harm without undermining their agency? This tension is especially visible in conversational systems, where safeguards can sometimes feel more paternalistic than supportive.
In my latest piece for Hugging Face, I argue that open source and community-driven approaches offer a promising (though not exclusive) way forward.
β¨ Transparency can make safety mechanisms into learning opportunities.
β¨ Collaboration with diverse communities makes safeguards more relevant across contexts.
β¨ Iteration in the open lets protections evolve rather than freeze into rigid, one-size-fits-all rules.
Of course, this isnβt a silver bullet. Top-down safety measures will still be necessary in some cases. But if we only rely on corporate control, we risk building systems that are safe at the expense of trust and autonomy.
Read the blog post here: https://huggingface.co/blog/giadap/preserving-agency
In my latest piece for Hugging Face, I argue that open source and community-driven approaches offer a promising (though not exclusive) way forward.
β¨ Transparency can make safety mechanisms into learning opportunities.
β¨ Collaboration with diverse communities makes safeguards more relevant across contexts.
β¨ Iteration in the open lets protections evolve rather than freeze into rigid, one-size-fits-all rules.
Of course, this isnβt a silver bullet. Top-down safety measures will still be necessary in some cases. But if we only rely on corporate control, we risk building systems that are safe at the expense of trust and autonomy.
Read the blog post here: https://huggingface.co/blog/giadap/preserving-agency
- 8 replies