Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • robber@lemmy.mlEnglish
    1·
    29 days ago

    You can control how much context should be fitted with --fit-ctx and how much space the algorithm should leave unallocated (even on a per-GPU basis) with --fit-target.