Why doesn't kdenlive load my 4 core CPU 100%?

Mr. Alex · 05-01-2014, 10:19 AM

Hi guys. When I render video in kdenlive on my Phenom X4 9650 I specify rendering for all 4 cores (selecting number of threads before hitting "Start rendering" button). But during rendering process all 4 cores are loaded about a half of their resource (as conky shows). Why does it happen? Why doesn't kdenlive use 100% of my CPU even if I make necessary configuration before processing?

PrinceCruise · 05-01-2014, 11:48 AM

Quote:

Originally Posted by Mr. Alex

Hi guys. When I render video in kdenlive on my Phenom X4 9650 I specify rendering for all 4 cores (selecting number of threads before hitting "Start rendering" button). But during rendering process all 4 cores are loaded about a half of their resource (as conky shows). Why does it happen? Why doesn't kdenlive use 100% of my CPU even if I make necessary configuration before processing?

You mean you set the CPU affinity using taskset?

Regards.

Mr. Alex · 05-01-2014, 02:11 PM

No, I mean this.

szboardstretcher · 05-01-2014, 02:18 PM

Quick test,.. if you select 1 cpu and time it, does it take 4x the time it would with all four cpu's selected?

Also, if you select 1 cpu, does it use 100% of the cpu? if you select two does it use 50% on each? if you select 4 does it use 25% a piece?

If so, it seems like the scheduling that they are using in the program itself is causing the odd behavior. Perhaps not suited for multiprocessor usage.

PrinceCruise · 05-02-2014, 03:34 AM

I may be wrong but I suspect this is the way Completely Fair Scheduler works in Linux. In my 4-core Intel i5, I use 2-3 VMs at the same time but not all the cores go to 100% at the same time. Most of the time they are used like 50-70% each.

Regards.

Mr. Alex · 05-02-2014, 04:46 AM

I use BFS. Does it behave the same way?

PrinceCruise · 05-02-2014, 04:58 AM

I have no idea how BFS works and what it's implementation is. May be you can try reaching it's maintainer.

Regards.

Mr. Alex · 05-02-2014, 05:19 AM

Quote:

Originally Posted by szboardstretcher

Quick test,.. if you select 1 cpu and time it, does it take 4x the time it would with all four cpu's selected?

Also, if you select 1 cpu, does it use 100% of the cpu? if you select two does it use 50% on each? if you select 4 does it use 25% a piece?

If I, for example, select 2 cores in that window, it doesn't make the situation any more clear. Estimated time becomes only about 1 minute more and CPU cores are loaded chaotically.

Quote:

Originally Posted by PrinceCruise

I have no idea how BFS works and what it's implementation is. May be you can try reaching it's maintainer.

I guess that's about the only way to find out.

syg00 · 05-02-2014, 05:56 AM

Very unlikely to be a scheduler issue. I regularly run multiple cores up to near 100% (CFS, not BFS).
If a multi-threaded application can't run cores to (near) 100%, it's usually because:
- it doesn't have the workload to need all those cores, or
- it isn't truly independent when threaded, some resource(s) - mutex, semaphore, shared memory, whatever - is in contention.

Better to talk to the applicaion support than scheduler - at least initially. Latencytop might help spot contention.

jpollard · 05-02-2014, 06:40 AM

Very likely each thread is on a different CPU... But each thread must also perform I/O, and during that delay, the CPU is idle, thus the 50% load.

Try setting to 6 threads... that way more I/O will be overlapped by processing done by other threads.

PS: I think your load should go to 75+%. You don't really want it to go much higher than that as you want to monitor the system as well.

TenTenths · 05-02-2014, 06:48 AM

Quote:

Originally Posted by jpollard

Very likely each thread is on a different CPU... But each thread must also perform I/O, and during that delay, the CPU is idle, thus the 50% load.

I'd tend to agree here. People tend to forget about IO requirements, also your disk, your choice of disks, your raid controller, raid level etc. will all make a difference. For example on a recent "real world" scenario we could decrease an 18+ hour import to 6 hours simply by changing from RAID5 to RAID1+0.

With any software/process it's rather simplistic to hope that using 4 threads will cause the process to run 4 times faster than 1 thread if there's any form of IO involved.

Mr. Alex · 05-02-2014, 06:51 AM

Quote:

Originally Posted by jpollard

Try setting to 6 threads... that way more I/O will be overlapped by processing done by other threads.

I can't do it. If I try to type in some number higher than 4, kdenlive doesn't allow me to do that (nothing is entered).

Mr. Alex · 05-02-2014, 06:53 AM

Quote:

Originally Posted by TenTenths

I'd tend to agree here. People tend to forget about IO requirements, also your disk, your choice of disks, your raid controller, raid level etc. will all make a difference. For example on a recent "real world" scenario we could decrease an 18+ hour import to 6 hours simply by changing from RAID5 to RAID1+0.

With any software/process it's rather simplistic to hope that using 4 threads will cause the process to run 4 times faster than 1 thread if there's any form of IO involved.

But during the rendering process my HDD led doesn't glow all the time. It blinks. Doesn't it mean HDD I/O isn't loaded 100% also?

jpollard · 05-02-2014, 02:59 PM

Quote:

Originally Posted by Mr. Alex

But during the rendering process my HDD led doesn't glow all the time. It blinks. Doesn't it mean HDD I/O isn't loaded 100% also?

It isn't - only about 50% of the time is it actually busy. And it may actually be less as the light takes time to come on, and time to go off after the signal comes in.

Mr. Alex · 05-03-2014, 02:31 AM

jpollard, then what part of PC is loaded 100% during this process?