C program has problem reading text file

woodeen · 01-09-2012, 10:20 AM

I run a certain hydrologic model that is written in C. I have compiled it with gcc on several versions of Debian and Ubuntu over the past five years and have never experienced the problem that I am about to describe.

When executed the program references a plain text configuration file that has a series of key pairs, where the name of the parameter is first specified, followed by the parameter value, separated by an equals sign (i.e., PARAMETER NAME = VALUE). Each key pair is on a separate line in the text file.

I recently replaced my hard drive with a new SSD, and at the same time upgraded to Ubuntu 11.10. I compiled the program in question with no errors noted. When I ran the same files that I had previously run under Ubuntu 11.04 the program told me that it could not find one of the key pairs. After trying many things I found that I had to remove all spaces between the parameter name and the parameter value (previously there could be multiple spaces in the text file). I ended up having to do this for all of the key pairs, but even so I still have at least one key pair that is not recognized even though I have removed all spaces either side of the equals sign.

I also tried running this same program under Debian Squeeze and Debian Wheezy (testing) and had the same problem. If I reinstall Ubuntu 11.04 on the same computer it works as it did before with no problems.

Thanks in advance for any suggestions about what the problem might be

trbennett48 · 01-09-2012, 06:32 PM

Sounds like a library routine has changed, but it
would be difficult to determine since we can only
guess what your source contains...

Might I suggest you "subset" your program, into a
version that only processes your input file and
demonstrates the problem... then post it, if you
haven't yet determined the problem.

-tony

woodeen · 01-10-2012, 12:21 PM

Thanks Tony for your reply. I have to admit that this is pushing the bounds of my understanding (I am an end user, not a programmer). In searching the source code for the main model routine I note the following which I believe are references to library files?

/******************************************************************************/
/* INCLUDES */
/******************************************************************************/
#include <time.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "settings.h"
#include "constants.h"
#include "data.h"
#include "DHSVMerror.h"
#include "functions.h"
#include "fileio.h"
#include "getinit.h"
#include "DHSVMChannel.h"
#include "channel.h"

Nominal Animal · 01-10-2012, 05:39 PM

Without any access to the source code that does the configuration parsing, how the hell do you expect us to give you an answer? Sure, we can guess, but without the sources that is all we can do.

That said, my guesses would be either locale differences affecting the configuration parsing (in which case changing LC_ALL and LANG environment variables should change the program behaviour), or when compiling, you had different libraries available in the two cases; perhaps it fell back to more restricted parsing for newer distributions, because some configuration parsing library is not installed. (The libraries installed by default do vary.)

If you still have both distributions installed, compare the locales (LC_ALL and LANG environment variables) first, and see if using the same affects the program behaviour.

Check if the program comes with documentation that describes its library dependencies, especially anything containing conf, cfg, or config in its name. If it uses autotools, see if the configuration command (./configure --help) lists, or tries to detect, some sort of a configuration library.

Or, perhaps the older distribution was 32-bit, and you also switched to a 64-bit one, and happened to uncover a bug in the programs' configuration parsing.

Only the source code for the configuration parsing parts would tell, but I for one am not going to register with my personal details (including street address) just to help you.

woodeen · 01-10-2012, 06:22 PM

I appreciate your help on this, and apologize for not being more helpful. Here is a link to the source code: ftp://ftp.hydro.washington.edu/pub/H...hsvm3.0.tar.gz

jthill · 01-10-2012, 07:47 PM

File "GetInit.c" has multiple instances of strcpy applied to overlapping strings, which was always undefined and in recent versions of glibc simply doesn't work. The one in "Strip" looks likeliest to me. At line 251, change

Code:

  strcpy(Buffer, StrStart);

to

Code:

  memmove(Buffer, StrStart, StrEnd+2-StrStart);

recompile and try it again. If that works, you might want to report it to the program's author -- in fact, you might want to check with the program's author whether this bug has already been fixed.

**** Edit: Originally had StrEnd+1, should be StrEnd+2 because he's got StrEnd pointing to the byte _before_ the string-ending null. ***

woodeen · 01-12-2012, 02:53 PM

Thank you for your reply, but unfortunately that did not solve the problem. I am pasting the terminal output from running the make file in hopes that there might be a clue to the problem?

cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o AdjustStorage.o AdjustStorage.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o Aggregate.o Aggregate.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o AggregateRadiation.o AggregateRadiation.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcAerodynamic.o CalcAerodynamic.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcAvailableWater.o CalcAvailableWater.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcBagnold.o CalcBagnold.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcDistance.o CalcDistance.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcEffectiveKh.o CalcEffectiveKh.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcKhDry.o CalcKhDry.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcKinViscosity.o CalcKinViscosity.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcSafetyFactor.o CalcSafetyFactor.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcSatDensity.o CalcSatDensity.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcSnowAlbedo.o CalcSnowAlbedo.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcSolar.o CalcSolar.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcTopoIndex.o CalcTopoIndex.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcTotalWater.o CalcTotalWater.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcTransmissivity.o CalcTransmissivity.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CalcWeights.o CalcWeights.c
CalcWeights.c: In function ‘CalcWeights’:
CalcWeights.c:171:28: warning: ‘closest’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o Calendar.o Calendar.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CanopyResistance.o CanopyResistance.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o ChannelState.o ChannelState.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CheckOut.o CheckOut.c
CheckOut.c: In function ‘CheckOut’:
CheckOut.c:109:8: warning: ‘l’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o CutBankGeometry.o CutBankGeometry.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o DHSVMChannel.o DHSVMChannel.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o Desorption.o Desorption.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o DistSedDiams.o DistSedDiams.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o Draw.o Draw.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o EvalExponentIntegral.o EvalExponentIntegral.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o EvapoTranspiration.o EvapoTranspiration.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o ExecDump.o ExecDump.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o FileIOBin.o FileIOBin.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o FileIONetCDF.o FileIONetCDF.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o Files.o Files.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o FinalMassBalance.o FinalMassBalance.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o FindValue.o FindValue.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o GetInit.o GetInit.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o GetMetData.o GetMetData.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InArea.o InArea.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitAggregated.o InitAggregated.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitArray.o InitArray.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitConstants.o InitConstants.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitDump.o InitDump.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitFileIO.o InitFileIO.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitFineMaps.o InitFineMaps.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitInterpolationWeights.o InitInterpolationWeights.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitMetMaps.o InitMetMaps.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitMetSources.o InitMetSources.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitModelState.o InitModelState.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitNetwork.o InitNetwork.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitNewMonth.o InitNewMonth.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitParameters.o InitParameters.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitSedTables.o InitSedTables.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitSedMap.o InitSedMap.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitSnowMap.o InitSnowMap.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitTables.o InitTables.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitTerrainMaps.o InitTerrainMaps.c
InitTerrainMaps.c: In function ‘InitVegMap’:
InitTerrainMaps.c:238:7: warning: the address of ‘VegMapFileName’ will always evaluate as ‘true’ [-Waddress]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitUnitHydrograph.o InitUnitHydrograph.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InitXGraphics.o InitXGraphics.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o InterceptionStorage.o InterceptionStorage.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o IsStationLocation.o IsStationLocation.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o LapseT.o LapseT.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o LookupTable.o LookupTable.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o MainDHSVM.o MainDHSVM.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o MainMWM.o MainMWM.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o MakeLocalMetData.o MakeLocalMetData.c
MakeLocalMetData.c: In function ‘MakeLocalMetData’:
MakeLocalMetData.c:238:16: warning: ‘WeightSum’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o MassBalance.o MassBalance.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o MassEnergyBalance.o MassEnergyBalance.c
MassEnergyBalance.c: In function ‘MassEnergyBalance’:
MassEnergyBalance.c:190:60: warning: array subscript is below array bounds [-Warray-bounds]
MassEnergyBalance.c:201:58: warning: array subscript is below array bounds [-Warray-bounds]
MassEnergyBalance.c:270:24: warning: ‘LD_FallVelocity’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o MassRelease.o MassRelease.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o MaxRoadInfiltration.o MaxRoadInfiltration.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o NoEvap.o NoEvap.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o RadiationBalance.o RadiationBalance.c
RadiationBalance.c: In function ‘RadiationBalance’:
RadiationBalance.c:153:19: warning: ‘Tau’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o ReadMetRecord.o ReadMetRecord.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o ReadRadarMap.o ReadRadarMap.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o ReportError.o ReportError.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o ResetAggregate.o ResetAggregate.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o RootBrent.o RootBrent.c
RootBrent.c: In function ‘RootBrent’:
RootBrent.c:215:1: warning: control reaches end of non-void function [-Wreturn-type]
RootBrent.c:199:60: warning: ‘e’ may be used uninitialized in this function [-Wuninitialized]
RootBrent.c:160:18: warning: ‘c’ may be used uninitialized in this function [-Wuninitialized]
RootBrent.c:164:7: warning: ‘b’ may be used uninitialized in this function [-Wuninitialized]
RootBrent.c:189:38: warning: ‘a’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o Round.o Round.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o RouteChannelSediment.o RouteChannelSediment.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o RouteRoad.o RouteRoad.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o RouteSubSurface.o RouteSubSurface.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o RouteSurface.o RouteSurface.c
RouteSurface.c: In function ‘RouteSurface’:
RouteSurface.c:326:10: warning: ‘DR’ may be used uninitialized in this function [-Wuninitialized]
RouteSurface.c:478:6: warning: ‘SedIn’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SatVaporPressure.o SatVaporPressure.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SensibleHeatFlux.o SensibleHeatFlux.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SeparateRadiation.o SeparateRadiation.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SizeOfNT.o SizeOfNT.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SlopeAspect.o SlopeAspect.c
SlopeAspect.c: In function ‘ElevationSlope’:
SlopeAspect.c:515:15: warning: ‘direction’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SnowInterception.o SnowInterception.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SnowMelt.o SnowMelt.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SnowPackEnergyBalance.o SnowPackEnergyBalance.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SoilEvaporation.o SoilEvaporation.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o StabilityCorrection.o StabilityCorrection.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o StoreModelState.o StoreModelState.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o SurfaceEnergyBalance.o SurfaceEnergyBalance.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o UnsaturatedFlow.o UnsaturatedFlow.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o VarID.o VarID.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o WaterTableDepth.o WaterTableDepth.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o channel.o channel.c
channel.c: In function ‘channel_read_network’:
channel.c:583:22: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
channel.c:606:17: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o channel_grid.o channel_grid.c
channel_grid.c: In function ‘Find_Next_Segment’:
channel_grid.c:239:29: warning: ‘NextAspect’ may be used uninitialized in this function [-Wuninitialized]
channel_grid.c:282:33: warning: ‘NextPtr’ may be used uninitialized in this function [-Wuninitialized]
channel_grid.c:266:53: warning: ‘CurrPtr’ may be used uninitialized in this function [-Wuninitialized]
channel_grid.c:239:29: warning: ‘CurrentAspect’ may be used uninitialized in this function [-Wuninitialized]
channel_grid.c: In function ‘Find_First_Segment’:
channel_grid.c:333:3: warning: ‘Ptr’ may be used uninitialized in this function [-Wuninitialized]
channel_grid.c: In function ‘RouteDebrisFlow’:
channel_grid.c:199:16: warning: ‘next_intj’ may be used uninitialized in this function [-Wuninitialized]
channel_grid.c:199:16: warning: ‘curr_intj’ may be used uninitialized in this function [-Wuninitialized]
channel_grid.c: In function ‘channel_grid_class’:
channel_grid.c:885:3: warning: ‘pntr’ may be used uninitialized in this function [-Wuninitialized]
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o equal.o equal.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o errorhandler.o errorhandler.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o globals.o globals.c
cc -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -c -o tableio.o tableio.c
cc AdjustStorage.o Aggregate.o AggregateRadiation.o CalcAerodynamic.o CalcAvailableWater.o CalcBagnold.o CalcDistance.o CalcEffectiveKh.o CalcKhDry.o CalcKinViscosity.o CalcSafetyFactor.o CalcSatDensity.o CalcSnowAlbedo.o CalcSolar.o CalcTopoIndex.o CalcTotalWater.o CalcTransmissivity.o CalcWeights.o Calendar.o CanopyResistance.o ChannelState.o CheckOut.o CutBankGeometry.o DHSVMChannel.o Desorption.o DistSedDiams.o Draw.o EvalExponentIntegral.o EvapoTranspiration.o ExecDump.o FileIOBin.o FileIONetCDF.o Files.o FinalMassBalance.o FindValue.o GetInit.o GetMetData.o InArea.o InitAggregated.o InitArray.o InitConstants.o InitDump.o InitFileIO.o InitFineMaps.o InitInterpolationWeights.o InitMetMaps.o InitMetSources.o InitModelState.o InitNetwork.o InitNewMonth.o InitParameters.o InitSedTables.o InitSedMap.o InitSnowMap.o InitTables.o InitTerrainMaps.o InitUnitHydrograph.o InitXGraphics.o InterceptionStorage.o IsStationLocation.o LapseT.o LookupTable.o MainDHSVM.o MainMWM.o MakeLocalMetData.o MassBalance.o MassEnergyBalance.o MassRelease.o MaxRoadInfiltration.o NoEvap.o RadiationBalance.o ReadMetRecord.o ReadRadarMap.o ReportError.o ResetAggregate.o RootBrent.o Round.o RouteChannelSediment.o RouteRoad.o RouteSubSurface.o RouteSurface.o SatVaporPressure.o SensibleHeatFlux.o SeparateRadiation.o SizeOfNT.o SlopeAspect.o SnowInterception.o SnowMelt.o SnowPackEnergyBalance.o SoilEvaporation.o StabilityCorrection.o StoreModelState.o SurfaceEnergyBalance.o UnsaturatedFlow.o VarID.o WaterTableDepth.o channel.o channel_grid.o equal.o errorhandler.o globals.o tableio.o -O3 -I. -Wall -Wno-unused -I/usr/local/include -I/usr/local/i386/include -o DHSVM -lm -L/usr/X11R6/lib -lX11

jthill · 01-12-2012, 04:29 PM

Darn.

Ok, so, I'd say the next steps are

Use the workaround for now (manually strip leading spaces from keywords & values in the init file)
Start talking to the developer about this bug, and point out the problem with the strcpy's

If using the workaround is beyond just inconvenient and the developer doesn't respond, post a self-contained test-case and I'll put some time into this.

I think getting hold of the developer is going to be important: if the code has this problem, it's possible it has others with more insidious symptoms (like, say, wrong answers) that need domain expertise to even identify.

Nominal Animal · 01-13-2012, 01:26 AM

Quote:

Originally Posted by woodeen

I run a certain hydrologic model

Do you have a small example input configuration and dataset -- even a trivial one -- we could use to test? I could easily write a replacement for the getinit.h/GetInit.c code; it uses a very standard config format, and I do believe I have something that could be trivially adapted for DHSVM.

woodeen · 01-25-2012, 08:32 PM

Sorry for the delay in getting back on this. Here is a link to a data set that will work with the model. There is a configuration file in the "input" directory, along with all other files needed to run the model. You will need to create an output location, and will have to adjust the paths in the configuration file to point to the input and output locations. Thank you for looking into this

jthill · 01-26-2012, 11:25 AM

Try changing -O3 in your makefile's CFLAGS to -O2? On my old box, it reads the parameter file correctly at any optimization setting, but segfaults later at -O3 and at least seems to run fine at -O2, I did the first two of the twelve years in that config file. It behaves this way with my patch or without, so my compiler (gcc version 4.6.2 (Debian 4.6.2-11)) or hardware (an old AMD64) don't do whatever is causing your parameter file problems. -O3 is ~start from the presumption that this program is perfect and make the code run as fast as you can~, and optimizers (a) can draw some surprising inferences these days and (b) aren't always perfect themselves.

woodeen · 03-15-2013, 11:35 AM

It has been over a year since the last post on this thread, but I want to follow up with what I think is the solution (sort of).

My workaround last year was to just go back to the last LTS version of Ubuntu and use that because I didn't have the problems that I had had with the more recent version of Ubuntu or Debian. When Ubuntu 12.04 LTS came out I switched to that and had no problems. More recently the model developers have released a new version of the model , which works fine on 12.04 LTS. I also tried this new model version on Debian Wheezy and it worked fine. Just this week I had reason to use a variant of the older version of the model; it worked fine on Ubuntu 12.04, but I had the same key-pair problem with Debian Wheezy! I spent a few hours trying to isolate what was different between the two models and found that a sub-program had been slightly rewritten in the newer model version. When I substituted that sub-program into the old version of the model the old version ran fine on Debian Wheezy. I still don't understand why the old model runs on Ubuntu 12.04 but not on Wheezy, but there you have it.

I am including the code form the sub-program below in case anyone is interested. Areas of bold text are new or modified code in the latest version.

Thank you all that took the time to try and help me with this.

Quote:

/*
* SUMMARY: GetInit.c - Get initialization data from file
* USAGE: Not a stand-alone program
*
* ORIG-DATE: 6-May-97 at 09:10:10
* DESCRIPTION: Functions are designed to behave more or less the same as the
* GetPrivateProfileString and GetPrivateProfileInt functions
* that are part of MFC (the functions here are a little less
* general). The functions search in a specified file for the
* entry that is associated with a certain key.
* The file is assumed to be organized in the same way as
* windows .ini files, i.e. the file consists of sections and
* key-value pairs. In addition there can be comments. Thus
* input files have the format:
*
* # comment
* [section]
* key=value # comment
* .
* .
* .
*
* DESCRIP-END.
* FUNCTIONS: GetInitString()
* GetInitLong()
* GetInitDouble()
* LocateKey()
* LocateSection()
* Strip()
* CopyDouble()
* CopyFloat()
* CopyInt()
* CopyLong()
* CopyShort()
* CopyUChar()
* IsEmptyStr()
* ReadInitFile()
* CreateNode()
* DeleteList()
* CountLines()

* Modification:
* $Id: GetInit.c, v 4.0 2012/10/31 Ning Exp $
* Comments:
*
*/
#define _CRT_SECURE_NO_DEPRECATE
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "DHSVMerror.h"
#include "fileio.h"
#include "getinit.h"

unsigned long GetInitString(const char *Section, const char *Key,
const char *Default, char *ReturnBuffer,
unsigned long BufferSize, LISTPTR Input)
{
LISTPTR SectionHead = NULL;

if ((SectionHead = LocateSection(Section, Input)) == NULL) {
strncpy(ReturnBuffer, Default, BufferSize);
return (unsigned long) strlen(ReturnBuffer);
}

if (!LocateKey(Key, ReturnBuffer, SectionHead)) {
strncpy(ReturnBuffer, Default, BufferSize);
return (unsigned long) strlen(ReturnBuffer);
}

return (unsigned long) strlen(ReturnBuffer);
}

/*#####################################################################################*/
long GetInitLong(const char *Section, const char *Key, long Default,
LISTPTR Input)
{
LISTPTR SectionHead = NULL;
char Buffer[BUFSIZE + 1];
char *EndPtr = NULL;
long Entry;

if ((SectionHead = LocateSection(Section, Input)) == NULL) {
return Default;
}

if (!LocateKey(Key, Buffer, SectionHead)) {
return Default;
}

Entry = strtol(Buffer, &EndPtr, 0);
if (EndPtr == Buffer) {
return Default;
}

return Entry;
}

/*#####################################################################################*/
double GetInitDouble(const char *Section, const char *Key, double Default,
LISTPTR Input)
{
LISTPTR SectionHead = NULL;
char Buffer[BUFSIZE + 1];
char *EndPtr = NULL;
double Entry;

if ((SectionHead = LocateSection(Section, Input)) == NULL) {
return Default;
}

if (!LocateKey(Key, Buffer, SectionHead)) {
return Default;
}

Entry = strtod(Buffer, &EndPtr);
if (EndPtr == Buffer) {
return Default;
}

return (Entry);
}

/*#####################################################################################
This function is used to find the matching key word in the input file for the "key"
specified in the fucntion: InitVegTable( )
#####################################################################################*/
unsigned char LocateKey(const char *Key, char *Entry, LISTPTR Input)
{
unsigned char Found = FALSE;
char Buffer[BUFSIZE + 1];
char KeyBuffer[BUFSIZE + 1];
char EntryBuffer[BUFSIZE + 1];
char *StrPtr = NULL;

/* Find a key in the current section */

if (Input) {
strncpy(Buffer, Input->Str, BUFSIZE);
while (!IsSection(Buffer)) {

/* Check whether the current line contains a key-entry pair */

if (IsKeyEntryPair(Buffer)) {
StrPtr = strchr(Buffer, SEPARATOR);
*StrPtr = '\0';
/* strcpy(KeyBuffer, Buffer); */
memmove(KeyBuffer, Buffer, strlen(Buffer)+1);
++StrPtr;
/* strcpy(EntryBuffer, StrPtr); */
memmove(EntryBuffer, StrPtr, strlen(StrPtr)+1);
Strip(KeyBuffer);
MakeKeyString(KeyBuffer);
if (strcmp(Key, KeyBuffer) == 0) {
Found = TRUE;
Strip(EntryBuffer);
/* strcpy(Entry, EntryBuffer); */
memmove(Entry, EntryBuffer, strlen(EntryBuffer)+1);
break;
}
}
/* Get the next line */
Input = Input->Next;
if (Input)
strncpy(Buffer, Input->Str, BUFSIZE);
else
break;
}
}

return Found;
}

/*#####################################################################################*/
LISTPTR LocateSection(const char *Section, LISTPTR Input)
{
char Buffer[BUFSIZE + 1];
char *StartPtr = NULL;
char *EndPtr = NULL;

while (Input) {
strncpy(Buffer, Input->Str, BUFSIZE);
if (IsSection(Buffer)) {
if (Buffer[0] == OPENSECTION) {
StartPtr = &Buffer[1];
EndPtr = strchr(Buffer, CLOSESECTION);
*EndPtr = '\0';
/* strcpy(Buffer, StartPtr);*/
memmove(Buffer, StartPtr, strlen(StartPtr)+1);
Strip(Buffer);
MakeKeyString(Buffer);
if (strcmp(Section, Buffer) == 0) {
Input = Input->Next;
break;
}
}
}
Input = Input->Next;
}

return Input;
}

/*#####################################################################################*/
unsigned char IsKeyEntryPair(char *Buffer)
{
char *StrSeparator = NULL;

StrSeparator = strchr(Buffer, SEPARATOR);
if (StrSeparator == NULL)
return FALSE;

return TRUE;
}

/*#####################################################################################*/
unsigned char IsSection(char *Buffer)
{
char *StrEndSection = NULL;
char *StrStartComment = NULL;

if (Buffer[0] != OPENSECTION)
return FALSE;

StrStartComment = strchr(Buffer, OPENCOMMENT);

StrEndSection = strchr(Buffer, CLOSESECTION);
if (StrEndSection == NULL)
return FALSE;

if (StrStartComment != NULL && StrStartComment < StrEndSection)
return FALSE;

return TRUE;
}

/*#####################################################################################*/
void Strip(char *Buffer)
{
char *StrEnd = NULL;
char *StrStart = Buffer;

/* remove leading whitespace */

while (*StrStart != '\0' && isspace((int) *StrStart))
++StrStart;

/* remove comment */

StrEnd = strchr(Buffer, OPENCOMMENT);
if (StrEnd != NULL)
*StrEnd = '\0';

/* remove trailing whitespace */

StrEnd = &Buffer[strlen(Buffer)];
--StrEnd;
while (StrEnd >= StrStart && isspace((int) *StrEnd)) {
*StrEnd = '\0';
--StrEnd;
}

/* strcpy(Buffer, StrStart); */
/* commented out by Ning b/c source and destination overlap */
memmove(Buffer, StrStart, strlen(StrStart)+1);
}

/*#####################################################################################*/
void MakeKeyString(char *Buffer)
{
char Str[BUFSIZE + 1];
char *PtrStr = Str;
char *PtrBuffer = Buffer;

/* toupper( ) converts the Buffer to uppercase and strip multiple spaces */

while (*PtrBuffer != '\0') {

if (*PtrBuffer == '\t') { //ignore the tab space
PtrBuffer++;
}
else {
*PtrStr = (char) toupper(*PtrBuffer);
if (isspace((int) *PtrBuffer)) {
while (isspace((int) *PtrBuffer))
PtrBuffer++;
}
else {
*PtrStr = (char) toupper(*PtrBuffer);
PtrBuffer++;}
PtrStr++;
}

}
*PtrStr = '\0';
strcpy(Buffer, Str);
}
/*#####################################################################################*/
int CopyDouble(double *Value, char *Str, const int NValues)
{
char *EndPtr = NULL;
int i;

for (i = 0; i < NValues; i++) {
Value[i] = strtod(Str, &EndPtr);
if (EndPtr == Str)
return FALSE;
Str = EndPtr;
}

if (EndPtr && *EndPtr != '\0')
return FALSE;

return TRUE;
}

/*#####################################################################################*/
int CopyFloat(float *Value, char *Str, const int NValues)
{
char *EndPtr = NULL;
int i;

for (i = 0; i < NValues; i++) {
Value[i] = (float) strtod(Str, &EndPtr);
if (EndPtr == Str)
return FALSE;
Str = EndPtr;
}

if (EndPtr && *EndPtr != '\0')
return FALSE;

return TRUE;
}

/*#####################################################################################*/
int CopyInt(int *Value, char *Str, const int NValues)
{
char *EndPtr = NULL;
int i;

for (i = 0; i < NValues; i++) {
Value[i] = (int) strtol(Str, &EndPtr, 0);
if (EndPtr == Str)
return FALSE;
Str = EndPtr;
}

if (EndPtr && *EndPtr != '\0')
return FALSE;

return TRUE;
}

/*#####################################################################################*/
int CopyLong(long *Value, char *Str, const int NValues)
{
char *EndPtr = NULL;
int i;

for (i = 0; i < NValues; i++) {
Value[i] = strtol(Str, &EndPtr, 0);
if (EndPtr == Str)
return FALSE;
Str = EndPtr;
}

if (EndPtr && *EndPtr != '\0')
return FALSE;

return TRUE;
}

/*#####################################################################################*/
int CopyShort(short *Value, char *Str, const int NValues)
{
char *EndPtr = NULL;
int i;

for (i = 0; i < NValues; i++) {
Value[i] = (short) strtol(Str, &EndPtr, 0);
if (EndPtr == Str)
return FALSE;
Str = EndPtr;
}

if (EndPtr && *EndPtr != '\0')
return FALSE;

return TRUE;
}

/*#####################################################################################*/
int CopyUChar(unsigned char *Value, char *Str, const int NValues)
{
char *EndPtr = NULL;
int i;

for (i = 0; i < NValues; i++) {
Value[i] = (unsigned char) strtol(Str, &EndPtr, 0);
if (EndPtr == Str)
return FALSE;
Str = EndPtr;
}

if (EndPtr && *EndPtr != '\0')
return FALSE;

return TRUE;
}

/*#####################################################################################*/
int IsEmptyStr(char *Str)
{
if (Str == NULL)
return TRUE;
if (Str[0] == '\0')
return TRUE;
return FALSE;
}

/*#####################################################################################*/
void ReadInitFile(char *TemplateFileName, LISTPTR * Input)
{
FILE *InFile = NULL; /* File with input information */
char Buffer[BUFSIZE + 1]; /* Tempora */
int i; /* counter */
int NLines; /* Number of lines in the input file */
LISTPTR Current = NULL; /* pointer to current node in list */
LISTPTR Head = NULL; /* pointer to the start of the list */

OpenFile(&InFile, (char *) TemplateFileName, "r", FALSE);

NLines = CountLines(InFile);
rewind(InFile);

for (i = 0; i < NLines; i++) {
fgets(Buffer, BUFSIZE, InFile);
Strip(Buffer);
if (IsSection(Buffer) || IsKeyEntryPair(Buffer)) {
if (Head == NULL) {
Head = CreateNode();
Current = Head;
*Input = Head;

}
else {
Current->Next = CreateNode();
Current = Current->Next;

}
strncpy(Current->Str, Buffer, BUFSIZE);

}
}

fclose(InFile);

return;
}

/*#####################################################################################*/
LISTPTR CreateNode(void)
{
LISTPTR NewNode = NULL;

NewNode = calloc(1, sizeof(INPUTSTRUCT));
if (NewNode == NULL)
ReportError("CreateNode", 1);
NewNode->Next = NULL;

return NewNode;
}

/*#####################################################################################*/
void DeleteList(LISTPTR Head)
{
LISTPTR Current = NULL;

Current = Head;
while (Current != NULL) {
Head = Head->Next;
free(Current);
Current = Head;
}

return;
}

/*#####################################################################################*/
int CountLines(FILE * InFile)
{
int NLines = 0;
char Buffer;

while ((Buffer = fgetc(InFile)) != EOF) {
if (Buffer == '\n')
NLines++;
}

return NLines;
}

/*#####################################################################################*/