Let's assume I have 2 GB of data stored on an NL115's CF card and haven't collected any of the data via LoggerNet yet. When I download the 2 GB via the Connect client, will all the data end up in the LN cache? If not, what happens to the "first in" data when the cache fills up?
I remember reading something about the cache being twice the size of the CR1000 memory. I'm curious if the same rule applies when a CF card is being used, i.e., will the cache for a CR1000 w/ 2 GB CF card be 4 GB?
My primary motivation for asking this question is to understand how these data will propagate from a CF card through the LN cache and on to a SQL Server DB via LNDB. If the LNDB service isn't running when I do the download, will I be able to update the DB after the data has finished downloading to the cache, or am I required to have LNDB running while the download is happening so that it can "keep pace" with the data download and not lose records as the "first in" data is overwritten?
Good question -- not something I have considered.
First thing to note is that the memory in the data cache is what we consider "ring memory". This means that once the data cache fills up, new records will begin overwriting the oldest records. If those old records that are overwritten are not yet in LNDB (or saved to file) then they will be lost.
I did a quick test in the datalogger with a data table that includes CardOut, a 256 M card, and various configurations for the Size parameters in both the DataTable instruction and the CardOut instruction. I used the Describe-Table command in LoggerNet's Cora Script to get information back on the table. This command returns the number of records allocated in the server's cache and the number of records allocated in the datalogger.
With the data table at 1000 and CardOut at -1, the cache value returned was 2,621,440 and the logger value returned was 62,824,356.
With the data table at -1 and CardOut at -1, the cache value was 2,621,440 and the datalogger value was 62,824,356.
With the data table at 1000 and the CardOut at 1000, the cache value returned was 2,621,440 and the logger value was 1008 (remember that card memory is allocated in "chunks", so the the number of records is really an "at least 1000 records").
With the data table at -1 and no card out, the cache value was 2,261,440 and the logger value was 830,297.
With the data table at 1000 and no card out, the cache value was 2,261,440 and the logger value was 1000.
And finally, two data tables at -1 and no card out, the first table was set to 2,621,440 and 414,706 and the 2nd table was set at 829,412 (cache) and 414,706 (logger).
So... all the above is probably more than you care to read, but I wanted to understand how the availability of a card was affecting the size of the cache in LoggerNet. The short answer is, it appears it does not.
In regards to LNDB, this would mean that if LNDB were not running, then there would be the potential of losing data. Note, however, that LNDB runs as a service, so if it is installed and the service is running (i.e., you haven't stopped it for any reason), then there should be no data loss (I haven't tested this, so this is based on my understanding of how it should work).
You also have the ability in LNDB to import data files, though using LNDB this is a manual operation (which could possibly be automated with a script, but I'm not an SQL expert to suggest that), which would be inconvenient if you needed to do it for a lot of data files on a regular basis.
I hope this helps. I will point our LoggerNet server engineer to this thread and ask him to make sure I haven't overlooked or misinterpreted anything.
Dana
As a follow-up to Dana's response: if you do run into problems you can use the corascript programming tool that allows "back-door" access to Loggernet to over-ride the default cache size for any station and table if you wish.
Help on Corascript (as a PDF or CHM file) can be found in here normally:
C:\Program Files\Campbellsci\LoggerNet
The resize-table command allows you to set the cache size of the cache for a specific table in a particular logger.
The LoggerNet server's approach to allocating storage in cache table files has become somewhat complicated. The original approach was to multiply the datalogger reported table size by the device tableSizeFactor setting (which defaults to two). Unfortunately, this approach brought us trouble on machines with limited storage space and could also drastically slow down the act of mounting very large files (LoggerNet pre-allocates the space for the cache table files). The solution to this was to introduce a new device setting, maxCacheTableSize, that specifies the maximum size, in bytes, of any cache table file created for that datalogger. This setting, by default, limits files to be no larger than 10 megabytes. There is also code in place in the LoggerNet server to shrink this even further if there is not enough disc space to store the requested size.
As Andrew stated above, the original size that the server allocated for the table can be adjusted using corascripts's resize-table command. Before doing so, however, you may need to adjust the cap defined by the maxCacheTableSize setting.