C++ Example 2: Reading data in text (.txt) format.

Ejemplo 2 para C++: Lectura de datos en formato .txt

After more than a month of not having updated this section of the blog (the others are being updated regularly though), this entry will show how to read .txt data using C++.

The input data for rainfall-runoff and environmental models is commonly provided by external sources, such as data loggers or the responsible of the IT department at the meteorological institutions, which rewrite the collected data in a certain format. Such data is often saved into binary (.bin) or text (.txt) format.  The former (.bin) is more efficient because it is the mother langauge of our computers. Data written in .bin format can be stored using less space than any other format, and can be retrieved at high speed; however, I have faced some situations where the writers of the .bin file forgot to write some details about the format of the data (e.g., is the data type Integer, Float, Double Long?), which made the data retrieval very difficult to accomplish . That is why throughout this example I am presenting some lines written in C++ that can be used to read a .txt file, a format that is popular among some groups of users.

Thus, the example shows:

i) how to read .txt data in C++, and

ii) how to format the input data into the format that our model uses.

A detailed explanation accompanies each line of the sample program, because I want to provide a practical example for those engineers interested in learning the C++ programming language.

Requirements.- When trying to read a .txt file, it is a must to perfectly understand the format of the file: Spaces matter as much as any alphanumeric character (see Figure 1).

Fig. 1 How different do the machine and you read (see) a text file?

The input data.- "File" type; it can be opened in a txt. format (e.g., using Note Pad when the file size is small, otherwise using some other software package such as the UltraEdit Text Editor); you (a human) can also read, and understand, the contents of the data contained in the file, but when the information is contained in a binary file, it can only be understood by the machine (you, the human, would only read some strange symbols).

The routine to read the txt. file.- At this time I am reading the data into an output file, assuming that the data contained will be used for a subsequent calculation in the body of the main program.

Let´s read the meteorological data for august, provided by a certain source, into the program. 

#include<stdio.h>
#include<stdlib.h>

int days=31;              //31days for August

char meteo[20]="meteorologicaldata.august";               
// name of the input file, with [20] characters.
// It is not the input file itself; it is only an auxiliary identifier named "fame", type char.
// Notice that it is not type "int" or "float" because we are not talking about the contents of the file;
// we are only talking about the name to which we will refer during this portion of the program.
// In other words, this line assigns the name of the input file "meteorologicaldata.august"
// to a shorter name "meteo".

char meteoout[25]="meteodata_august.txt";                   // output file
// we read it out into a variable named, e.g.,  meteoout. Again, we are not reading the data of the information
// contained into "meteodata_august.txt";
// we are merely assigning a shorter name and a variable to meteodata_august.txt.
// Thus, we could have simply written char meteodata_august.txt if wished.
 
//The following lines are the variables.
// In brackets are written the sizes of the matrices to which the data will be written to.
//[31][24] means a matrix with 31 rows (days in august) and 24 hours (for a day).
// Notice the difference between that assignation and the assignation in the lines above.

unsigned short raina[31][24],rainb[31][24];      // raina is the rain at station "a"; same for rainb
short tempa[31][24],tempb[31][24];                // tempa is the air temperature at station "a"; same for tempb
int i,j;

// now begins the main routine
void main()
{
   // let´s imagine that the input file has also the code of the station, written with two characters [2]
    unsigned short stcode[2];

   // let´s imagine that besides rain and temperature, the station has recorded the date,
   // and the station type (stntype)
    short date[3],sttype[1];

    // let´s imagine that the station name is represented by a single character [1]
    // let´s introduce some additional auxiliar vectors for temporal storage of the rain and the temp: m, n
    short stn[1],m[1],n[1];

    // the following are the pointers that will call the input and the outputs
    // the fp is a general pointer
    FILE *meteo,*meteoout,*fp;
   
    /******* below we will read the meteorological data file *****************/
    // the following "if" routine reads the file in binary format (rb, or read binary)
    // you may use this "error proof" format when reading any input data,
    // because it will send you a message (e.g. error reading file) when there is some problem with it
    if((finame=fopen(meteo,"rb"))==NULL){       
        printf("error reading file \n");
        exit(1);
    }
    // the following "if" routine opens the output file that will contain the meteorological data
    // in text format (w, or write text)

    if((foutame=fopen(meteoout,"w"))==NULL){      
        printf("error reading file \n");
        exit(1);
    }

    // Inicialization of rain[31][24] and temp[31][24] for the two stations

    for(i=0; i<=30; i++){
        for(j=0; j<=23; j++){
            raina[i][j]=32767;
            rainb[i][j]=32767;
            tempa[i][j]=32767;
            tempb[i][j]=32767;
        }
    }
   
    // The following buckle reads the data from the meteorological data file,
    // element by element, and later writes the data down to the output files
    while(1){
        // Lest´s begin reading from the 1st to the 4th element
        fread(stn,2,1,meteo);   
         // station name : (the variable where the data is read, size in bytes of the element to be read,
         //                        number of elements with size "size", the file that contains the input data)
        // e.g., the variable stn is type short, which means it has a size of 2 bytes

        fread(stcode,2,2,meteo);    // station code
        fread(sttype,2,1,meteo);    // station type
        fread(date,2,3,meteo);        // date yy/m/d
       
        // Let´s imagine that all data for a single day, 24hrs, is written on a single row
        for(i=0; i<=23; i++){   
            // let´s imagine that rain, and temp are the 5th, 6th elements
            // let´s read them into auxiliary vectors m and n


            fread(m,2,1,meteo);    // reading rain; 2 bytes, 1 element
            fread(n,2,1,meteo);    // reading temp
           
            // now we write the temporary vectors into the raina[31][24],rainb[31][24]
            // which are the final vectors for the output 
            // Let´s imagine 11111=code for station A; 22222=code for station B
            if(stcode[0]==11111) raina[date[2]-1][i]=m[0];
            if(stcode[0]==22222) rainb[date[2]-1][i]=m[0];
 
            // similarly, now we write them down into tempa[31][24],tempb[31][24]
            // which are the final vectors for the output file
            if(stcode[0]==11111) tempa[date[2]-1][i]=n[0];
            if(stcode[0]==22222) tempb[date[2]-1][i]=n[0];
        }

       
        // end of reading the file with the meteorological data
         //The next step is to write it down to the output text file       
Please contact me if you need the remaining part of the program. I will only need your name, the use you intent to give to the program, and your compromise to cite the source. The reason to ask that is that I have produced this very simple program during my stay at Tohoku, and I do not want to violate any copyrights. Thanks. Tweet