Using The IDL DataMap Library With New Format Files

Original Author: R. J. Barnes (JHU/APL)

Overview

This tutorial shows you how to use the datamap.pro library to work with fitacf and rawacf files. Although the SuperDARN IDL libraries contain routines for reading and writing these files, you can also use the datamap.pro library to do this directly, and in some cases this may be a simpler approach.

This tutorial assumes that you already know how to work with DataMap files and are familiar with the concepts involved. The DataMap format is documented here, and a tutorial on using IDL to work with DataMap files is here. It also assumes that you know the variables stored in rawacf and fitacf files. The RFC#0008 describes the rawacf format and RFC#0009 describes the fitacf format.

Reading fitacf Files

The example program presented below does the simple task of reading records from a fitacf file and printing out the velocity measurements on the console:

pro readfitacf

  ; Open input file for reading
 
  openr,unit,'test.fitacf',/get_lun
  
  while  DataMapRead(unit,sclvec,arrvec) ne -1 do begin
 
     if (n_elements(sclvec) ne 0) then begin
        yrid=DataMapFindScalar('time.yr',2,sclvec)
        moid=DataMapFindScalar('time.mo',2,sclvec)
        dyid=DataMapFindScalar('time.dy',2,sclvec)
        hrid=DataMapFindScalar('time.hr',2,sclvec)
        mtid=DataMapFindScalar('time.mt',2,sclvec)
        scid=DataMapFindScalar('time.sc',2,sclvec)
        usid=DataMapFindScalar('time.us',2,sclvec)
        nrangid=DataMapFindScalar('nrang',2,sclvec)
    endif
  
    if (n_elements(arrvec) ne 0) then begin
       slistid=DataMapFindArray('slist',2,arrvec)
       vid=DataMapFindArray('v',4,arrvec)
    endif
   
      if (yrid eq -1) or (moid eq -1) or (dyid eq -1) or (hrid eq -1) $
         or (mtid eq -1) or (scid eq -1) or (usid eq -1) or (nrangid eq -1) $
         or (slistid eq -1) or (vid eq -1) then begin
        print,'File is in the wrong format!'
        stop
    endif

    ; get the scalar values
 
      yr=*sclvec[yrid].ptr
      mo=*sclvec[moid].ptr
      dy=*sclvec[dyid].ptr
      hr=*sclvec[hrid].ptr
      mt=*sclvec[mtid].ptr
      sc=*sclvec[scid].ptr
      us=*sclvec[usid].ptr
      nrang=*sclvec[nrangid].ptr

     ; get the list of ranges

      slist=*arrvec[slistid].ptr
      
      v=fltarr(nrang)

      ; get the velocities and put them in the right elements of v

      v[slist]=*arrvec[vid].ptr
      
      print, yr,mo,dy,hr,mt,sc,us
      print, nrang
      print, v

      s=DataMapFreeScalar(sclvec)
      s=DataMapFreeArray(arrvec)
 
  endwhile


   free_lun,outp
end

The program extracts the velocity information and populates the array v. The problem with fitacf files is that depending on whether a good fit was obtained, a range may not be stored in the file. The array variable slist contains a list of the ranges that were stored. To correctly populate the velocity array we use the following:

slist=*arrvec[slistid].ptr
v=fltarr(nrang)
v[slist]=*arrvec[vid].ptr

Reading and Writing rawacf Files

The example program presented below reads records from a rawacf file, plots the ACF of the first valid range, scales the ACF's and writes them to a new rawacf file.

pro readrawacf

  ; Open input file for reading

  sclname=['nrang','mplgs']
  arrname=['slist','acfd']
  scltype=[2,2]
  arrtype=[2,4]

  openr,inp,'input.rawacf',/get_lun
  openw,out,'output.rawacf',/get_lun
  
  while  DataMapRead(inp,sclvec,arrvec) ne -1 do begin

     sclid=[-1,-1]
     arrid=[-1,-1]

     if (n_elements(sclvec) ne 0) then begin
        for n=0,n_elements(sclname)-1 do $
          sclid[n]=DataMapFindScalar(sclname[n],scltype[n],sclvec)
     endif
   

    if (n_elements(arrvec) ne 0) then begin
       for n=0,n_elements(arrname)-1 do $
          arrid[n]=DataMapFindArray(arrname[n],arrtype[n],arrvec)
   endif

    q=where(sclid eq -1,count)
    if (count ne 0) then begin
        print,'File is in the wrong format!'
        stop
    endif


    q=where(arrid eq -1,count)
    if (count ne 0) then begin
        print,'File is in the wrong format!'
        stop
    endif

    nrang=*sclvec[sclid[0]].ptr
    mplgs=*sclvec[sclid[1]].ptr

    ; get the list of ranges

   slist=*arrvec[arrid[0]].ptr
            
   acfd=fltarr(2,mplgs,nrang)
       
     
   ; get the velocities and put them in the right elements of v

   acfd[*,*,slist]=*arrvec[arrid[1]].ptr

   rng=slist[0]
   plot,acfd[0,*,rng]
   oplot, acfd[1,*,rng]

   acfd=0.8*acfd    
      
   *arrvec[arrid[1]].ptr=acfd[*,*,slist] 

   s=DataMapWrite(out,sclvec,arrvec)
   if (s eq -1) then begin
     print, 'Error writing file'
     stop
   endif

   s=DataMapFreeScalar(sclvec)
   s=DataMapFreeArray(arrvec)
 
  endwhile

   free_lun,inpp
   free_lun,outp
end

This program uses a slightly simpler method of determining that the file contains the required information. As with the fitacf file, the variable slist contains the list of ranges that contain data. To correctly populate the velocity array we use the following:

slist=*arrvec[arrid[0]].ptr
acfd=fltarr(2,mplgs,nrang)
acfd[*,*,slist]=*arrvec[arrid[1]].ptr

When we have finished processing the data, we must perform the inverse mapping before writing the output file:

acfd=0.8*acfd    
*arrvec[arrid[1]].ptr=acfd[*,*,slist] 
s=DataMapWrite(out,sclvec,arrvec)