How can I limit ffmpeg concurrency

1

When my users upload video server gets loaded too much because of plenty of ffmpegs …
How can I limit ffmpeg concurrency

2

Hi zdv777 ,

Thanks for the question , what you mean is your users are uploading many video files to server then these actions made your server overloaded , right ? and you want to limit the users stop upload video.

Thanks.

3

No, I don’t want to limit users.
Each user uploads video and a preview is generated for each video.
FFmpeg task is scheduled for each preview generation and executed right away.
If there are too much videos uploaded, too much ffmpegs are executed sumultaneously.

What I want is to limit scheduled task execution concurrency eg 100 users upload 100 videos at a time and 100 ffmpeg tasks are scheduled, but only 8 tasks are executed concurrently. Extra tasks should wait.

4

What kind of server are you using for Razuna? We run ffmpeg without any
restrictions and have over 5000 users on our hosted platform.

5

Hi Nitai,
How many ffmpegs are running at once on your servers at top load?

I use X8DAH dual X5680 24Gb Gentoo x64
When lots of local users upload lots of large videos to archive server via multiple gigabit networks files are uploaded much faster than previews are calculated and number of running ffmpegs grow eating CPU power, disk IO speed and also some RAM. At average load about 60 server responds rather slowly. Too slowly to be honest.
I think CPU and IO consuming operations like ffmpeg rendering should be queued.
At this time I’m thinking about substituting ffmpeg executable with script, which should control the queue.
The only drawback I see in this method is possibility to run beyond maximum processes limit.