Archive overview

The DSP stores its main persistent resources in-memory in the Starcounter database, which has great performance benefits when working with short-term data. It’s, for example, extremely fast to add new database rows and query the in-memory database. It’s, however, not optimal for storing huge amounts of seldom accessed data over time, as this often requires more RAM memory than can be justified for this type of data. For this, a disk-based archive database is what we want, where we can offload data entities in order to free up memory.

For example, every time a bid request is received from the Mopedo backend and evaluated by the DSP, a new row is added to a table in the Starcounter database that the DSP is connected to. This means that the web activity generated by the client’s customers is recorded over time, but can also make the in-memory database table holding bid requests too large to manage reliably. To deal with the potentially hundreds of millions of bid requests accumulating in the database, the Mopedo DSP can move old bid requests to a SQLite based bid request archive resource called Mopedo.Archive.BidRequestArchive. Since the SQLite database is not kept in-memory, it can contain a very large number of rows without affecting the overall performance of the DSP.

Archive resources

The Mopedo.Archive namespace contains the following five archive resources:

Routines

The archiving process, which includes copying entities to archive resources and optionally deleting entities from the in-memory resources, is controlled by means of routines. Routines are scheduled operations that can archive and/or delete entities when there are either too many of them, or when they are older than a certain time.

Routines are managed using the Mopedo.Archive.Routine resource.