• 0 Posts
  • 6 Comments
Joined 5 months ago
cake
Cake day: October 15th, 2024

help-circle


  • I would be cautious to trust an American company just because they are non-profit. We know that if they get sufficiently big they can just transition into being for profit (see OpenAi), chances are regulations regarding this won’t become stronger later.

    Signal being open source of course helps, but it’s usually the ecosystem that people grow to rely on that keeps them in place, not the technology. Just look at how big Reddit is compared to Lemmy for a good example of that. If signal was federal that would be quite different since jumping ship would be easy.



  • Arkthos@pawb.socialtoPrivacy@lemmy.worldPlease, don't!
    link
    fedilink
    English
    arrow-up
    6
    ·
    17 days ago

    You can offload them into ram. The response time gets way slower once this happens, but you can do it. I’ve run a 70b llama model on my 3060 12gb at 2 bit quantisation (I do have plenty of ram so no offloading from ram to disk at least lmao). It took like 6-7 minutes to generate replies but it did work.