Greene's Programming Pages
C++

I learned C++ from the beginning of my learning C/C++, and the two books I used to learn it from both steered me away from using C's FILE*, fgets, fread, fwrite, and so on. (In fact, neither Bruce Eckel — whose C++ teaching I have come to love very dearly — nor John T. Berry even mention at all how to do file i/o with C's FILE*.) Liking the iostream and fstream of C++, I never had any occasion to look "backward" at the C way of doing it. Until now.

I've been using Borland's C++ compiler 5.02 for the last few years. I generates pretty efficient code. Over the last several months (Jan-Sep, 2000), I have begun using the relatively new STL structures (nice stuff, that!), and as I get into using it more and more, I have noticed that the Borland 5.02 compiler sometimes generates some slow-running EXEs, compared to the Borland 5.5 compiler. I have shied away from the Borland 5.5 compiler specifically because its fstream file i/o is much less efficient than the 5.02 compiler's fstream file i/o. This prompted me to give Microsoft's Visual C++ 6.0 compiler a try (I'm using the current 6.0 version). I have noticed with various programs I've written that with my programs that used a lot of STL, the VC++ compiler generated faster EXEs, but with programs that had little or no STL the Borland 5.02 compiler was better.

Then, I tried out this one program of mine that uses an STL map structure to count the occurring values in a field in a fixed- record text file, and discovered that the Borland 5.02 compiler (using fstream) made a much more efficient EXE than the VC++ compiler, meaning that this EXE's time was being spent mostly on just reading the file, and very little on counting the occurring values in the map structure.

This motivated me to try a little test. I changed the file reading (I was using the fstream getline function) to C's FILE* and fgets.

WOW!!!

The difference is truly amazing! Under all compilers the EXE is substantially more efficient. And I found that the VC++ generated an EXE that was about 15%-20% faster than the Borland 5.02 compiler. And all of this simply by dumping fstream and using FILE*. What a lesson for me!

My data processing program users are sometimes processing hundreds of thousands, or even the occasional millions of record, and these substantial time differences can really add up and make a difference over time, which is why I dug into this. So if you write programs that need to perform a lot of file i/o, and you're using C++'s fstream, you might want to check into going "backward" a bit. It could be worth it.

Here is some detailed information on what I've tested showing the distinct differences in file i/o efficiency between C++'s fstream and C's FILE*.

The stripped down file i/o test program that I used is below.

This program simply performs a read on the input file using either the fstream "getline" function or the FILE* "fgets" function (which by default reads characters up to the next hex 0A [linefeed] into the buffer), checks to see if there is a hex 0D [carriage return] at the end of the line in the buffer (since reading in binary mode) and decrements the "len_buf" if so, then writes "len_buf" characters from the buf to the output file and adds a hex 0A [linefeed] on the end of the line. (This could actually be used to convert a PC text file to a UNIX text file because it converts the CRLF line-delimiter to simply a LF line-delimiter.)

As you can see, the vast majority of the work is simple file i/o, with very little else going on.

My system happens to have a Pentium II chip at 266 MHz. For generating times, I used an input file consisting of exactly 99,696 fixed-length CRLF-delimited records of 217 characters (219 bytes/record counting the CRLF on the end), which is a 21,833,424 byte file. I have two physically separate hard drives on my PC, and when I ran the program to "clock" the times, I had my source file on one drive and my destination file going to the other drive.

The three compilers I have are Borland 5.02 (B50), Borland 5.5 (B55), and (Microsoft) Visual C++ 6.0 (VC). Here are the times I recorded (in seconds):

C++:  B50  B55   VC
      ---  ---  ---
       14   34   28
       13   33   29
       13   34   28
       13   33   28
       14   33   28

C:    B50  B55   VC
      ---  ---  ---
       13   12   11
       12   13   13
       13   12   11
       13   12   11
       13   13   12

Now, MFC's CFile actually uses C code, so if you're using MFC you're getting the more efficient FILE* file i/o automatically. I could understand some small "performance hit" with using the C++ fstream (say, 10% or so), but it is the more than 100% performance hit that strikes me as being a bit ridiculous. Borland's 5.0 compiler's fstream, as you can see from my results, had a very tiny performance hit (if any at all, since the difference in time shown here is so small so that it could in fact be attributable to simple fluctuation). So what happened with Borland 5.5 that Borland killed the efficient fstream?

And for those of you who might be thinking, "Yeah, yeah, who cares about 10 or 25 seconds?" I want to point out that if you happen to be running, say, 10 MILLION records instead of just 100,000, then you are talking about the difference between, say, 20 minutes using FILE* and 50 minutes using fstream, meaning that if you use fstream you could be adding 30 minutes of completely unnecessary time for your program to run just performing basic file i/o. That's a big difference. These things add up substantially.

I would be curious to hear from others regarding their own results with this test program, especially in regard to relative times using other compilers as well, such as a GNU C++ compiler (with the libraries to make Win32 executables) or Watcom. It would also be interesting to hear from those of you who compile on UNIX platforms, to see if there are any substantial differences in file i/o times with compilers there between fstream and FILE*.

(By the way, I'm running a Windows NT 4.0 platform. The majority of the programs I write are for the purpose of processing large amounts of "record type" data (like what you might do with COBOL on a mainframe), so processing times with large numbers of records is a priority for me. As you can see, the program above is strictly a "console app" that would run in a console window on Windows NT or Windows 9x. If you're on a UNIX platform (say, Linux or AIX or anything like that), the code above should be compatible (even though I'm on Windows, I try to stay with compatible - more-or-less standard - C++ source code) though you'll probably have to alter the "include" files a bit.)

A C++ programmer,
Todd S. Greene
<tgreene@usxchange.net>
(Sep. 13, 2000)