LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 03-27-2009, 07:57 AM   #1
10110111
Member
 
Registered: Jun 2008
Location: St.-Petersburg, Russia
Distribution: (B)LFS, Ubuntu, SliTaz
Posts: 403

Rep: Reputation: 51
Open software to benchmark GCC vs. MSVC code generation


What software can i use to compare its performance when it's compiled with GCC and when compiled with MSVC? I think it should be some mathematical software, so the performance would be very dependent on CPU- and memory-specific optimizations.
 
Old 03-28-2009, 10:09 AM   #2
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
What are you hoping to learn? I might have some direct answers to your original question (actual open source projects that would yield interesting performance results) depending on your answer. (The rest of this post does not directly answer your actual question, but it might help you).

You will get very different results depending on which programs you choose to test. You will get very different results depending on which options you give on the compile commands. Both GCC and MSVC can be tuned significantly by skilled use of their badly documented performance options.

Are you doing all this in win32 or win64?

The Win32 architecture is seriously short of registers. GCC is fundamentally a generic (architecture independent) compiler. It has many win32 specific tweaks, but it still does very badly in many examples because it isn't well designed for an architecture with a serious register shortage. MSVC was tuned for win32 before win64 even existed and it does a decent job with the register shortage, often resulting in far better code than gcc can manage. The old Intel 7.1 compiler is even better at dealing with that register shortage and consistently generates much better code than MSVC for win32.

The Win64 architecture works pretty well for a compiler that is mainly generic and has a few architecture specific tweaks. So the fundamental disadvantage of gcc vs. MSVC or Intel is gone. In win64 you are more likely to see gcc generating better code than those others.

Also, what version of gcc?

On linux I have extensively tested several versions of gcc (with a lot of effort put into tuning command line switches) against the Intel 10 compiler. If you just pick an optimization level (don't carefully tune command line switches) Intel 10 will consistently beat gcc (Intel takes less compile time and produces code that runs faster). But the Intel 10 compiler responds badly to any attempt to improve it further by tweaking command line switches. None of them do what the doc says they do, few non default settings change anything at all vs. default, and if they change anything they're more likely to make it worse. GCC command line options can be used wisely. Up through version 4.1.2 that still wasn't enough to be competitive with Intel 10. But starting in version 4.3.2 I have gcc generating better code than Intel 10 in many examples.

On Win64, Intel 10 consistently beats MSVC in code quality. I'm also using gcc 4.4 from the 64bit Mingw, but I haven't yet compared its code quality to either MSVC or Intel 10.

Last edited by johnsfine; 03-28-2009 at 10:12 AM.
 
Old 03-28-2009, 03:55 PM   #3
10110111
Member
 
Registered: Jun 2008
Location: St.-Petersburg, Russia
Distribution: (B)LFS, Ubuntu, SliTaz
Posts: 403

Original Poster
Rep: Reputation: 51
Quote:
You will get very different results depending on which options you give on the compile commands.
Yes, i know this. My question was in some way generic, i.e. i asked about software to play with its source code using different command-line switches, but such software should be CPU&memory intensive and compilable both in linux (GCC) and windoze (MSVC).
Quote:
Are you doing all this in win32 or win64?
You mean IA32 or IA64 (Win## usualy stands for windoze architecture)? Or, maybe you suppose i'm going to test GCC in cygwin under windoze? I do on IA32, for instance, Pentium 4, and i am going to try MSVC under windoze, and GCC under linux.
Quote:
Also, what version of gcc?
I have installed 4.1.2 as LFS of the time of compiling offered it, but i would like to test newer versions on another, binary distro.
 
Old 03-28-2009, 04:38 PM   #4
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
Originally Posted by 10110111 View Post
i asked about software to play with its source code
I'm sure you can find some open source software for data compression. Set to its slowest mode (maximum compression) data compression software tends to be a good test of the quality of code generation in a compiler.

Quote:
such software should be CPU&memory intensive and compilable both in linux (GCC) and windoze (MSVC).
I don't think you really want memory intensive. The code quality from the compiler is rarely any performance factor at all in a memory intensive task. The OS might be a big factor. The algorithm in the source code is probably an important factor. But the compiler is generally not a factor.

Also testing linux against windows at the same time you test gcc against msvc may be very misleading. Will you know which difference matters?

Quote:
You mean IA32 or IA64 (Win## usualy stands for windoze architecture)?
In that case I would have meant IA32 vs. x86_64. IA64 is a much less common architecture.

I did mean win32 vs. win64, because I assumed you were comparing compilers under equal conditions.

Quote:
Or, maybe you suppose i'm going to test GCC in cygwin under windoze?
I was hoping you meant win64, so I expected you would test 64bit Mingw against MSVC.

I don't know where you can get a decent version of gcc for win32 from either Mingw or Cygwin. I've only found fairly old lame versions of gcc for 32 bit Cygwin and Mingw.

BTW, are you using the expensive version of MSVC or the intentionally performance sabotaged free version?

Quote:
I do on IA32, for instance, Pentium 4, and i am going to try MSVC under windoze, and GCC under linux.

I have installed 4.1.2 as LFS of the time of compiling offered it, but i would like to test newer versions on another, binary distro.
In linux X86_64 the performance improvement in gcc generated code is significant from 4.1.2 to 4.3.2. I think 4.4 has even more performance improvements. But I don't know if any of those improvements have the same impact in IA32. Even in win64, the ABI has a significant difference from the X86_64 standard, which will impact the effectiveness of various choices the compiler makes. The developers of 64 bit Mingw are just looking for compliance with the win64 ABI while the developers of gcc look at x86_64 performance only for the standard ABI. With no one actively tuning gcc's win64 performance, there are likely flaws.

I'm still bothered by your choice of masking a compiler comparison with an OS comparison. If you manage to select CPU intensive programs where the quality of code generation matters and the memory accesses, the I/O and the math library are all less significant, then the OS can be kept from skewing the results. But that narrows the right answers to your original question and I'm not sure I know any.

BTW, the math library can skew things a LOT. Some of my own performance testing accidentally compared a GNU version of the exp() function with an Intel version. On some ranges of input the Intel version is so much faster that this difference dominates the entire result, even though I though I was comparing performance of a complex simulation of which calls to exp() were a minor part.
 
Old 03-28-2009, 10:03 PM   #5
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939
The operating environments are so very different... but anyhow, what you're asking for is "a benchmark, the hard way."
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
MSVC => GCC inline assembly translation rlanger Programming 1 02-03-2009 10:06 AM
A GCC code generation problem? dogbird Programming 4 12-09-2005 11:52 AM
tool for code generation? bcalmac Linux - Software 0 08-22-2005 10:41 AM
Code generation based on latex math ahwkong Programming 0 04-17-2005 06:28 AM
UML and C++ code generation GŠutama Linux - Software 2 11-13-2003 06:56 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:38 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration