Not the best on AI/LLM terms, but I assume that training the models was done on Nvidia, while inference (using the model/getting the data from the model) is done on Huawei chips
To add: Training the model is a huge single-cost expense, while inference is a continuous expense.
Are you by chance using an integrated GPU?
Noticed that my AMD Radeon 680M uses quite a lot of RAM as shared memory.
Using something like amdgpu_top will show how much RAM your iGPU is using, metric is ‘GTT’
The whole downside is that not everyone is a data horder with space for videos
Some media players allows for streaming directly using yt-dlp, e.g.;
mpv <youtube url>
Will use yt-dlp if installed
Not what I expected, good thing you managed to get it solved!
That dump didn’t reveal any particular useful information, however it seems like multiple people are reporting issues with mesa + segfault, e.g. https://bbs.archlinux.org/viewtopic.php?id=301550
Mesa v24.3.2-1 in Arch should revert that issue, Mesa v24.3.1 seems to be the problem one
You could check the backtrace of one of your crashes
coredumpctl debug
> bt
And then dump that trace here
It might be related to Mesa/GPU drivers
Sounds like you might just be max’ing out the capacity of the coax cable as well (depending on length/signal integrity). E.g. ITGoat (not sure how trustworthy this webpage is, just an example) lists 1 Gbps as the maximum for coax while you would typically expecting less than that, again depending on your situation (cable length, material, etc)
What’s your situation into the wall? Depending on country/ISP/regulations they might give you up to 1000 Mbps under the assumption that it’s a single line going to a single user, however quite often that line is shared with potentially a lot of different customers.
Some countries allows you to buy packages where you have a standalone line going to your wall, however at an additional cost
Yes, so R&D and finalizing the model weight is done on NVIDIA GPUs (I guess you need an excessive amount of VRAM).
Inference is probably gonna be offloaded to consumers in the end where the NPU is taking care of the inference cost (See Apple, Qualcomm etc)