Changes in CGNS 3.1

Time-dependent Connectivity

CPEX0027 has been implemented. The added functions to support this extension are described in Zone Grid Connectivity in the Mid-Level Library.

Zone Subregions

CPEX0030 has been implemented. The added functions to support this extension are described in Zone Subregions in the Mid-Level Library.

General SIDS Improvements

CPEX0031 has been implemented. The added functions to support this extension are described in Flow Solution and Discrete Data in the Mid-Level Library. There are also changes to Boundary Conditions and numerous places in the SIDS.

64-bit Implementation

You only need to read the following if you have a 64-bit computer and need to work with problems that exceed 2 billion coordinates and/or elements. Or, if you are bored.

C Interface

The major change in 3.1 is the introduction of the data type cgsize_t, which will be a 64-bit integer when building 64-bit code, and a 32-bit integer otherwise. This is defined in a new header file, cgnstypes.h, which is included in cgnslib.h. Some additional data types have also been introduced, cglong_t which is always 64-bits, cgint_t and cgerr_t which are 32-bit integers, and cgid_t which is a 64-bit floating point number (double). With the exception of cglong_t these other data types are currently not used. The cglong_t data type is used internally, and may be used by application code in conjunction with DataType_t LongInteger to write 64-bit data to the CGNS file, regardless of the compilation mode. The cgnstypes.h header file also defines 3 flags, which describe the current build mode, CG_BUILD_LEGACY, CG_BUILD_64BIT, and CG_BUILD_SCOPE, which are set to 0 or 1, depending on the build mode.

When the --enable-legacy option is given to configure, cgsize_t is defined as an int (integer*4), CG_BUILD_LEGACY is set to 1, and CG_BUILD_64BIT and CG_BUILD_SCOPE are set to 0. The --enable-scope and --enable-64bit options to configure are ignored. This will generate the same code as for the versions prior to 3.1.

When the --enable-64bit option is given to configure, cgsize_t is typedef'd as long, long long or __int64 (depending on the machine) and the equivalent FORTRAN data type is integer*8. All other values are unchanged. This will set CG_BUILD_64BIT to 1, and CG_BUILD_LEGACY to 0 in cgnstypes.h. The value of CG_BUILD_SCOPE depends on the configure option below.
If this option is not given, then cgsize_t is typedef'd as int, and the equivalent FORTRAN data type is integer*4. The value of CG_BUILD_64BIT is set to 0.

When the --enable-scope option is given to configure, scoping of the enumeration values in cgnslib are enabled. This will prefix all the enums with CG_. This is disabled by default and when --enable-legacy is enabled. If enabled, this will set the value of CG_BUILD_SCOPE in cgnstypes.h to 1.

These options are also available for cmake users, and have the same behavior.

When building 64-bit code, the ADF file format changes, and thus will not be readable by prior versions of the library. All dimensions are stored as 64-bit integers, and the modifications to the ADF interface allow addressing and storage up to 1,000 terabytes of data. The HDF5 interface is also effected due to the the dimension sizes. I'm not sure of the HDF5 limitations to file size, but I expect it to be as large as that addressable by a 64-bit integer.

Thus, when writing a CGNS file in 64-bit mode it will only be readable by the version 3.1 software. A 32-bit compilation of version 3.1 will be able to read the 64-bit CGNS file, but will fail gracefully if the dimensions exceed those that can be handled by a 32-bit integer. With a 32-bit mode compilation, the ADF version and HDF5 interfaces are unchanged, and thus the CGNS files will be readable by earlier version software.

Regardless of how the version 3.1 software is compiled, it is backwards compatible with earlier version CGNS files.

A new data type has also been added to the CGNS MLL, LongInteger, which is defined in the DataType_t enumeration. This refers to a true 64-bit integer. Previous versions of the MLL did not explicitly allow for 64-bit integer data. This data type is automatically used for dimensions, connectivity, boundary condition points, holes and elements when compiled in 64-bit mode. It may also be used by an application to define array data as 64-bit, regardless of the compilation mode. In this case, the data should be defined as cglong_t in the code in order to guarantee 64-bit data being passed to the array read or write routines.

The following routines have been modified to use the cgsize_t data type. 

cg_zone_read(int fn, int B, int Z, char *zonename, cgsize_t *size);
cg_zone_write(int fn, int B, const char * zonename,
	const cgsize_t * size, CGNS_ENUMT(ZoneType_t) type, int *Z);

cg_coord_read(int fn, int B, int Z, const char * coordname,
	CGNS_ENUMT(DataType_t) type, const cgsize_t * rmin,
	const cgsize_t * rmax, void *coord);
cg_coord_partial_write(int fn, int B, int Z,
	CGNS_ENUMT(DataType_t) type, const char * coordname,
        const cgsize_t *rmin, const cgsize_t *rmax,
        const void * coord_ptr, int *C);

cg_section_read(int file_number, int B, int Z, int S,
	char *SectionName,  CGNS_ENUMT(ElementType_t) *type,
	cgsize_t *start, cgsize_t *end, int *nbndry, int *parent_flag);
cg_section_write(int file_number, int B, int Z,
	const char * SectionName, CGNS_ENUMT(ElementType_t) type,
	cgsize_t start, cgsize_t end, int nbndry, const cgsize_t * elements,
	int *S);
cg_section_partial_write(int file_number, int B, int Z,
	const char * SectionName, CGNS_ENUMT(ElementType_t) type,
	cgsize_t start, cgsize_t end, int nbndry, int *S);

cg_elements_read(int file_number, int B, int Z, int S,
	cgsize_t *elements, cgsize_t *parent_data);
cg_elements_partial_write(int fn, int B, int Z, int S,
	cgsize_t start, cgsize_t end, const cgsize_t *elements);
cg_elements_partial_read(int file_number, int B, int Z, int S,
	cgsize_t start, cgsize_t end, cgsize_t *elements, cgsize_t *parent_data);

cg_parent_data_write(int file_number, int B, int Z, int S,
	const cgsize_t * parent_data);
cg_parent_data_partial_write(int fn, int B, int Z, int S,
	cgsize_t start, cgsize_t end, const cgsize_t *ParentData);

cg_ElementDataSize(int file_number, int B, int Z, int S,
	cgsize_t *ElementDataSize);
cg_ElementPartialSize(int file_number, int B, int Z, int S,
	cgsize_t start, cgsize_t end, cgsize_t *ElementDataSize);

cg_field_read(int fn, int B, int Z, int S, const char *fieldname,
	CGNS_ENUMT(DataType_t) type, const cgsize_t *rmin,
        const cgsize_t *rmax, void *field_ptr);
cg_field_partial_write(int fn, int B, int Z, int S,
	CGNS_ENUMT(DataType_t) type, const char * fieldname,
	const cgsize_t *rmin, const cgsize_t *rmax,
        const void * field_ptr, int *F);

cg_hole_info(int fn, int B, int Z, int I, char *holename,
	CGNS_ENUMT(GridLocation_t) *location,  CGNS_ENUMT(PointSetType_t) *ptset_type,
	int *nptsets, cgsize_t *npnts);
cg_hole_read(int fn, int B, int Z, int I, cgsize_t *pnts);
cg_hole_write(int fn, int B, int Z, const char * holename,
	CGNS_ENUMT(GridLocation_t) location, CGNS_ENUMT(PointSetType_t) ptset_type,
	int nptsets, cgsize_t npnts, const cgsize_t * pnts, int *I);

cg_conn_info(int file_number, int B, int Z, int I,
	char *connectname, CGNS_ENUMT(GridLocation_t) *location,
	CGNS_ENUMT(GridConnectivityType_t) *type,
	CGNS_ENUMT(PointSetType_t) *ptset_type,
	cgsize_t *npnts, char *donorname,
	CGNS_ENUMT(ZoneType_t) *donor_zonetype,
	CGNS_ENUMT(PointSetType_t) *donor_ptset_type,
        CGNS_ENUMT(DataType_t) *donor_datatype,
	cgsize_t *ndata_donor);
cg_conn_read(int file_number, int B, int Z, int I, cgsize_t *pnts,
        CGNS_ENUMT(DataType_t) donor_datatype,
	cgsize_t *donor_data);
cg_conn_write(int file_number, int B, int Z,
	const char * connectname, CGNS_ENUMT(GridLocation_t) location,
	CGNS_ENUMT(GridConnectivityType_t) type,
	CGNS_ENUMT(PointSetType_t) ptset_type,
	cgsize_t npnts, const cgsize_t * pnts, const char * donorname,
	CGNS_ENUMT(ZoneType_t) donor_zonetype,
	CGNS_ENUMT(PointSetType_t) donor_ptset_type,
        CGNS_ENUMT(DataType_t) donor_datatype,
        cgsize_t ndata_donor, const cgsize_t *donor_data, int *I);
cg_conn_write_short(int file_number, int B, int Z,
	const char * connectname, CGNS_ENUMT(GridLocation_t) location,
	CGNS_ENUMT(GridConnectivityType_t) type,
	CGNS_ENUMT(PointSetType_t) ptset_type,
	cgsize_t npnts, const cgsize_t * pnts, const char * donorname, int *I);
cg_conn_read_short(int file_number, int B, int Z, int I,
	cgsize_t *pnts);

cg_1to1_read(int fn, int B, int Z, int I, char *connectname,
	char *donorname, cgsize_t *range, cgsize_t *donor_range, int *transform);
cg_1to1_write(int fn, int B, int Z, const char * connectname,
	const char * donorname, const cgsize_t * range,
	const cgsize_t * donor_range, const int * transform, int *I);
cg_1to1_read_global(int fn, int B, char **connectname,
	char **zonename, char **donorname, cgsize_t **range,
	cgsize_t **donor_range, int **transform);

cg_boco_info(int fn, int B, int Z, int BC, char *boconame,
	CGNS_ENUMT(BCType_t) *bocotype, CGNS_ENUMT(PointSetType_t) *ptset_type,
 	cgsize_t *npnts, int *NormalIndex, cgsize_t *NormalListFlag,
 	CGNS_ENUMT(DataType_t) *NormalDataType, int *ndataset);
cg_boco_read(int fn, int B, int Z, int BC, cgsize_t *pnts,
	void *NormalList);
cg_boco_write(int file_number, int B, int Z, const char * boconame,
	CGNS_ENUMT(BCType_t) bocotype, CGNS_ENUMT(PointSetType_t) ptset_type,
	cgsize_t npnts, const cgsize_t * pnts, int *BC);

cg_array_info(int A, char *ArrayName,
	CGNS_ENUMT(DataType_t) *DataType,
	int *DataDimension, cgsize_t *DimensionVector);
cg_array_write(const char * ArrayName,
	CGNS_ENUMT(DataType_t) DataType, int DataDimension,
	const cgsize_t * DimensionVector, const void * Data);

cg_ptset_info(CGNS_ENUMT(PointSetType_t) *ptset_type,
	cgsize_t *npnts);
cg_ptset_write(CGNS_ENUMT(PointSetType_t) ptset_type,
	cgsize_t npnts, const cgsize_t *pnts);
cg_ptset_read(cgsize_t *pnts);

Note that, as in earlier versions of the library, the donor_datatype argument to cg_conn_write and cg_conn_read are redundant. Earlier versions required this value to be Integer. In 3.1 this is ignored altogether, and is taken to Integer in 32-bit mode and LongInteger in 64-bit mode. The corresponding donor_data is always of type cgsize_t.

A new convenience function has been added to the MLL: 

cg_index_dim(int file_number, int B, int Z, int *index_dim)
which gets the index dimension for the specified base and zone. The return value will be the Cell Dimension for a structured grid, and 1 for an unstructured grid.

C Portability

If you use the cgsize_t data type in new code, it will work in both 32 and 64-bit compilation modes. In order to support CGNS versions prior to 3.1, you may also want to add something like this to your code: 
#if CGNS_VERSION < 3100
#define cgsize_t int
#endif
Existing code that uses int will not work with a CGNS 3.1 library compiled with the enable-64bit option turned on. You may want to add something like this to your code: 
#if CGNS_VERSION >= 3100 && CG_BUILD_64BIT
#error does not work in 64 bit mode
#endif
or modify your code to use cgsize_t.

FORTRAN Interface

All integer arguments in the FORTRAN interface are taken to be integer*4 in 32-bit mode and integer*8 in 64-bit mode. If you have used default or implicit integers in your FORTRAN code, it should port to 64-bit mode in most cases by simply turning on your compiler option that promotes implicit integers to integer*8. However, be sure that you read the next section about issues. For example, all the FORTRAN code in the User's Guide examples ported unmodified when using the -fdefault-integer-8 compiler option with gfortran. If you have explicitly defined your integers as integer*4, your code will not work in 64-bit mode. In that case, you will either need to change them to integer (recommended for portability) or integer*8.

The cgnslib_f.h header file has been modified to explicitly define all integer values based on the CGNS library compilation mode, integer*8 when compiled in 64-bit mode and integer*4 otherwise. A new integer parameter has also been added to the header, CG_BUILD_64BIT, which will be set to 1 in 64-bit mode and 0 otherwise. You may use this parameter to check if the CGNS library has been compiled in 64-bit mode or not, as in: 

if (CG_BUILD_64BIT .ne. 0) then
    print *,'will not work in 64-bit mode'
    stop
endif
If you are using a CGNS library prior to version 3.1, this parameter will not be defined and you will need to rely on your compiler initializing all undefined values to 0 (not always the case) for this test to work.

FORTRAN Issues and Portability

If your compiler supports automatic promotion of integers, and you use implicit integers, your code should port to 64-bit with the following exception.

If you use an Integer data type in any routine that takes a data type specification, and an implicit integer for the data, the code will fail when compiled in 64-bit mode with automatic integer promotion. An example of this would be: 

integer dim
integer data(dim)
call cg_array_write_f('array',Integer,1,dim,data)
This is because the MLL interprets the Integer data type as integer*4 regardless of the compilation mode. The compiler, however, has automatically promoted data to be integer*8. What you will need to do to prevent this problem, is to either explicitly define data as in: 
integer dim
integer*4 data(dim)
call cg_array_write_f('array',Integer,1,dim,data)
or 
integer dim
integer*8 data(dim)
call cg_array_write_f('array',LongInteger,1,dim,data)
or test on CG_BUILD_64BIT as in: 
integer dim
integer data(dim)
if (CG_BUILD_64BIT .eq. 0) then
  call cg_array_write_f('array',Integer,1,dim,data)
else
  call cg_array_write_f('array',LongInteger,1,dim,data)
endif
The last 2 options will only work with CGNS Version 3.1, since LongInteger and CG_BUILD_64BIT are not defined in previous versions. Note also, that this does not apply to the donor_datatype argument to cg_conn_write and cg_conn_read as discussed above.

You may also need to be careful when using integer constants as arguments in 64-bit mode. If your compiler automatically promotes integer constants to integer*8, then there is no problem. This is probably the case if your compiler supports implicit integer promotion. If not, then the constants will be integer*4, and your code will not work in 64-bit mode. In that case you will need to do something like: 

integer*8 one,dim
integer*4 data(dim)
one = 1
call cg_array_write_f('array',Integer,one,dim,data)
If you can't or don't want to use automatic integer promotion, but still want to be able to port to 64-bit code, you may include the preprocessor header file, cgnstypes_f.h in your code. In 64-bit mode, the CG_BUILD_64BIT define in the header will be set to 1, and cgsize_t and CGSIZE_T will be defined as integer*8, otherwise CG_BUILD_64BIT will be 0 and cgsize_t and CGSIZE_T will be defined as integer*4. By including this header file as a preprocessor include (i.e. #include "cgnstypes_f.h") you may then use cgsize_t as variable type name. for example: 
#include "cgnstypes_f.h"
cgsize_t cgfile,cgbase,celldim,physdim
cgsize_t one,dim,ierr
cgsize_t data(dim)

call cg_open_f('file.cgns',CG_MODE_WRITE,cgfile,ierr)
celldim = 3
physdim = 3
call cg_base_write_f(cgfile,'base',celldim,physdim,cgbase,ierr)
...
one = 1
#if CG_BUILD_64
  call cg_array_write_f('array',Integer,one,dim,data)
#else
  call cg_array_write_f('array',LongInteger,one,dim,data)
#endif
Notice, that you will not be able to use integer constants, but your FORTRAN code will be portable to 64-bit mode.

Lastly, you may use the header to simply limit the code to 32 or 64-bit modes, such as: 

#include "cgnstypes_f.h"
#if CG_BUILD_64BIT
# error can only be built in 32-bit mode
#endif

Examples

All the distributed C examples have been modified to use cgsize_t. and will work in both 32 and 64-bit modes. They are also fully compatible with previous CGNS versions.

C Test Program

There is a new C example in the tests subdirectory, test64c.c, that you may use for simple 64-bit mode testing. To compile it, do make test64 from within the tests subdirectory. The program allows you to write any number of coordinates or tetrahedral elements using whatever amount of memory you specify. To get the list of available options, enter: 
test64c -help
or
test64c --
Without any options, the program will try to use 16 billion bytes of memory, and write 4 billion float coordinates and 500 million tetrahedra. The number of coordinates and tetrahedra default to the number that will fit within the available memory, in this case (16b / 4) and (16b / 32). To specify the amount of memory, use the -m option. For example, to use 4 billion bytes, run as 
test64c -m4b
In this case, the program will write 1 billion coordinates and 125 million tetrahedra, by default. I recommend that you use less than the full amount of available memory, otherwise swapping will result in your elapsed run time going from minutes to hours (or even a day).

To specify the number of coordinates or elements to write, use the -c and -e options. The number may be followed by a qualifier (b for billion, m for million, or k for thousand). For example, to write 10 billion coordinates and 400 million tetrahedra using 2 billion bytes of memory, use: 

test64c -m2b -c10b -e400m
If the memory required for either the coordinate or element data exceeds the memory available, the program will use partial writes to write the data. In the case above, the coordinates will be written 500m and the elements 62.5m at a time. A value of 0 may also be used for the number of coordinates or elements, in which case they are not written. Be aware, that 1 billion coordinates and elements require a liitle over 4 Gb and 32Gb of disk space, respectively.

To control the output file format, use -adf to force ADF and -hdf5 for HDF5 output. The last option is -test, which will cause the file to be reopened in read-only mode and the data read and checked for accuracy.

FORTRAN

The examples cgread.F and cgwrite.F in the tests subdirectory have been modified to support 64-bit compilation. The cgnstype_f.h header was included in these, and the cgsize_t definition for variables was used as described above. This makes the code portable to 64-bit without requiring any additional compiler options.

Another example, test64f.F has been created in this directory. This program explicitly defines the integer variables as integer*8 and then tests on the CG_BUILD_64BIT variable in order to prevent the program from running (and failing) unless built in 64-bit mode.

These FORTRAN examples do not really require 64 bits to run, they are provided to help with porting of your code. To build these, do make fortran from within the tests subdirectory.