Further Lightroom beta musings

By way of an expansion upon yesterdays histrionics, I took it upon myself to offer some affirmation of the statistics by repeating the tests on an alternate platform. Said machine comprised:

Intel Core i7 940 @ 2.93GHz

6GB Triple Channel DDR3 @ 1600MHz

As previously, a single disk is in use, and photographs are imported using the "Add" option, so as to minimise the impact of disk IO on the tests.

If we compare first the relative performance of Lightroom 2.4 and 3 Beta (both 64bit builds), the pronounced inferior performance of 3 across the board is plain to see:

Thus affirming the statistics from yesterday:

Finally, it would appear that hyperthreading makes little or no difference. Multi thread support is an architectural consideration that cannot realistically be introduced outside of the formative stages of a project. Since Lightroom 3.0 is apparently not a green field development, we would not expect relative performance with HT on or off to differ in 3 where it did not in 2.

As regards the general performance, fair enough - it's a beta. I know very well myself that speed is a key concern at the beginning and end of the software life cycle, in terms of architectural decisions and then optimisation respectively. If Lightroom is going to perform poorly at any stage of its production, the time is now - but even so the difference is pronounced. Any judgement is premature because the specific nature of this build is not clear, but given that performance is one of the primary concerns for any user of a digital darkroom, we shall be keeping a very close eye on things as future builds become available.

Benchmarking LR 3.0 Beta

So we thought it would be interesting to benchmark the new Adobe Lightroom 3 beta. Clearly it's not finished, hence beta, but it's still interesting to see where Adobe are headed, no?

What's going to happen is that I am going to import 50 D3 raw files, render 1:1 previews, batch process the photos and then export to jpeg. So that's 3 tests. This will be carried out on my work pc in LR3, then again in LR2.4. This same process will also be followed on Adrian's i7 work pc (with and without Hyper Threading enabled), and as a result, we should have a general idea about i7 vs Core 2 on the new Lightroom, about the merits of hyperthreading (so is i7 worth it over i5) and finally about potential improvements from LR2 to LR3.

If you're lucky, we may even break out some graphs for y'all.

Disclaimers etc.

All tests are using the system drive. This just keeps things as even as possible across machines. Clearly if you have a set of RAIDed SSDs to do your bidding, things will get better; less flippantly, given we're looking at processors primarily here, the disk drives will bottleneck some of these tests (I'm looking at you 1:1 preview and exporting), but that will be revealed in the results.

I only did these tests once, honestly, I should be doing work now, so 3 or more to get a good average ain't gonna happen today.

This is using beta software, results are clearly no indication of final performance, and if you buy something off the back of our results and we're wrong, more fool you.

Phew!

To the details...

Simon's pc:

Windows 7 x64

Intel Core 2 Quad Q6600@2.4ghz

4GB ram

Nvidia Quadro NVS 290

Test 1:

The raw files are already on my pc's hard drive, and they're merely being added to the library. At the same time I'm rendering 1:1

LR 3.0 5:19.3

LR 2.4 3:24.9

Test 2:

in the first image

exposure +1.00

contrast +50

vibrance +30

saturation +30

sharpening 50

noise reduction/color 40

copy, choose all

switch to grid view, highlight photos 2-50 and right click any picture, and select develop, paste

LR 3.0 0:18.0

LR 2.4 0:05.7

Test 3:

select all files, file, export, jpeg, 100%, srgb, desktop folder

LR 3.0 4:53.7

LR 2.4 2:37.8

So yes. That's having the beta software running more slowly for me than the mature software. Surprised? You should know better ;)