What is the TSCache?

Like all caches, the TSCache was designed to enable rapid access to a very large set of information as if that information was all available from directly accessible memory (RAM), but without requiring an unduly large amount of RAM in order to operate -- keeping all but the most recently used information on disk/SSD. Also like other caches, this cache enables this information to be accessed through a set of relatively smaller “keys”, hiding the complexity of managing this information and providing access seamlessly.

The TSCache (like some but not all caches) offers concurrent, thread-safe access.

It may be “embedded” with an application executing on a single server or PC; or, it may be deployed in “client-server mode” (where multiple applications executing on other PCs or servers may access a common cache), or in “distributed” mode (whereby a single TSCache is replicated on multiple server instances providing for faster read access by a greater number of users and robustness in the event of server failure [i.e., replication]). It also offers restart capability (at a performance cost) and transactional capability (at a greater performance tradeoff).

How is the TSCache different from its competitors?

The TSCache offers 4 different types of interfaces for accessing data whereas competitors generally offer one. These interfaces are not just different ways of storing and accessing data, but provide for key functionality differences – e.g., multiple keys to a single data item and Big Data capabilities with multi-dimensional search. The different interfaces are discussed below.

The TSCache enables bulk operations that provide high performance when storing and retrieving multiple cached data items in a single call. These operations are often performed in parallel by this Cache -- providing unparalleled (if you'll forgive the pun) performance.

There are tremendous advantages that enable rapid concurrent access by multiple users, processes and/or threads. These advantages provide for extreme performance through the parallelization and concurrency aspects of the design and implementation of this Cache. Importantly: Provides for fair scheduling of resources when there are multiple processes/threads and full allocation of all resources (allocated cores) whether engaged by one or multiple processes.

The TSCache enables in-place modification of very large single data items (e.g., video or genetic sequences) thus obviating the need for retrieving and re-saving these items in their entirety during modifications.

Interfaces

We offer 4 Interfaces:

1. Ix (Index) interface
2. Key Cache Interface
3. Dictionary (Key-Value Pair) Interface
4. Tagged Ix Interface

3 of our 4 interfaces are unique. Other caches we are aware of offer only the Dictionary Interface. Some advantages of this cache is faster speed -- especially during concurrent access.

1. Ix Interface

Provides for unlimited array-like access to data;
Stores data and returns an integer index; then, enables retrieval of this data (or pieces of this data) using that index.

int allocateAndStore(byte [ ] dataOfAnySz) ….. store
byte [ ] retrieve(int ix) ….. retrieve
byte [ ] retrievePartial(int ix, int startPos, int endPos) ….. retrieve partial
void store(int ix, byte [] dataOfAnySz) ….. replace
void remove(int ix) ….. delete (free up space)
byte [] retrieveAndRemove(int ix)

Advantages:

Provides for faster storage and access than Dictionary and Key Cache Interfaces.
The retrievePartial( ) capability is extremely useful when the data stored is very large (e.g., a genetic sequence).

2. Key-Cache Interface

Ability to define Data Types and to retrieve property information from byte arrays stored.
Capability to define multiple keys for each datatype and ability to retrieve an item from any of multiple keys.

3. Dictionary (Value-Key) Interface

Key-value pair access -- similar to the manner that almost all commercial and open source caches work today.
Provides capability to drop this cache into existing code with minimal requirements for code change.

4. Ix Tagged Interface

A combination of Ix Interface with the ability to tag cached objects with one or more tags.
Capability of creating tag hierarchies – a key facilitator for data mining/Big Data analysis.
Provides log2-N performance multi-dimensional data access/recall for any combination of Tags.

Deployment

Embedded

Thread-safe: Enables multi-core creation of cached data.
Concurrent access: Enables multi-core use and analysis of large-scale data local to Purchaser’s code
Enables extreme performance in workstation-based applications

Client Server

Enables multiple client access to data cached on a single large server.

Distributed (Coming soon)

Enables deployment of a single cache executing across a grid of machines.
Replication: Provides performance, robust operation (protects against single server failure).
Data sharding: Enables data to be distributed among many execution engines – both for performance during insertion/modification and query execution performance.

Robustness

Transactional (Coming soon [Transactionality for single operations already provided -- see below.])

Enables transactionality across Cache Operations – single or multi (replicated) instances.
With Replication, guarantees transactionality across instances.

Always

Atomic Cache Operations – A single operation will either be completely done or undone on any single instance.

Performance is far higher than "Transactional" due to attention to detail in design of reversible steps if a single operation is not completed.
Transactional -- but only for a single cache operation (not across cache operations).

After Graceful Shutdown

As long as Cache is shutdown gracefully, all data will be intact when the cache is brought back up. Otherwise, cache will be brought up empty.
More than an order of magnitude faster than modes mentioned above.

Clean

Cache is always brought up as empty and starts fresh whether shutdown gracefully or not.
More than an order of magnitude faster modes mentioned above. (Same as "After Graceful Shutdown", so this is a choice.)

Performance

PBEF – Participatory Bulk Execution Framework

Bulk Operations – sub-divided and handled by a pool of a configurable number of worker threads
Revolutionary Scheduling Mechanism:
Enables tasks submitted later to share worker thread resources
Two-tier prioritization w/lower-tier still receiving a minimal configurable %age of resources during execution of higher priority tasks.
All users receive their fair share of resources regardless of when they submit (so quick operations do not wait for larger operations submitted prior to complete).
Participatory – dedicates invoking thread to task submitted
Submitting thread participates with some/full bulk worker thread participation to accomplish task.
Guarantees performance equal to or better than executing thread alone.
Work units within a bulk operation can be batched.
Participatory is optional – PBEF can be used w/passed callback for async submission w/all other benefits
User receives maximum performance possible given the total workload placed on the TSCache at any given time.

Attention to detail across nodes coupled with unique design elements lead to superior performance

In Transaction Restartable Mode: Unique replication design among instances – replicating via positional data vs. repetition of execution via replicated logs. (Coming soon!)
In Always Restartable Mode: Detailed reversible processes ensure that data is written to durable storage only at the end of steps where necessary to enable reversibility if atomic operation is not completed; and, importantly, not during other steps.
In AfterGracefulShutdown and in "Clean" Mode: Data is never forced to durable storage until the TSCache is logically closed.

Free Block Allocation Mechanism

Non-fragmentation model: Guarantees no performance degradation over time.
Pre-allocation of DataBlocks: Ensures space for new items can be obtained quickly.

Configuration Options to Optimize Performance – Configurable in accordance with characteristics of

Purchaser’s Data: Size and Distribution of elements of data
Purchaser's Hardware: Cores (number/speed), Memory and Type of Durable Storage
Purchaser's Usage: E.g., whether
Largely read-only with occasional writes vs. large turnover [like Market Data]
There will be concurrent and/or multi-threaded access vs. single user access, etc.

Ix Interface Performance

Faster than dictionary (key-value pair) access of industry-leading competing caches

Optimized for multi-user, multi-threaded and concurrent access

Peformance gap increases significantly vs. competitors when accessed by a large number of users/threads concurrently

Fastest performance for concurrent writes.

Big Data

Multi-Keyed

As with a database, ability to have multiple keys to access a single TSCache data item.
Index on keys for a data item updated concurrently.

Tagging

Ability to retrieve items tagged with any combination of tags (AND and OR)
Multi-dimensional performant search: Provides this capability with unique design enabling recall of items with combinations of tags in an indexed manner (without scanning)
Parent hierarchies of tags – enabling recall of all sub-tags of a parent with parent tag

Competitor Comparison

Functionality

No Competitors – except Berkeley DB (which, isn’t a Cache) support multiple keys on a single item.
No Competitors have an Ix Interface (although that can be simulated with a numeric key).
No Competitors support tagging and the multi-dimensional search that this provides.
No Competitors have the load-balancing capabilities of this Cache when accessed by multiple concurrent threads/users and none prevent resource starvation and provide for fair allocation (vs. a first-come, first-serve approach when multiple processes are retrieving/writing large amount of data)
No Competitors provide for modification of a piece of a single large item of data in the cache (e.g., a piece of a video or genetic sequence) without retrieving and re-saving the entire piece of data.

Performance

When Deployed in Always-Restartable Mode: (Cache data survives if server crashes.)
Single user/thread access: – Superior performance even during single process access.
35% to 40% faster than nearest competitor when performing large numbers of individual read and write operations by a single user.
400% to 500% faster than nearest competitor when performing bulk read and write operations by a single user (where a user submits many read or write requests to be performed in a single opeartion).
Multiple user access or when cache is accessed by many concurrent threads:
Many multiple times faster^* than nearest competitor when performing large numbers of individual read and write operations by multiple threads or users.
An even higher number of multiples faster^* than nearest competitor when performing bulk read and write operations by multiple threads and/or users (where a user/thread submits many read or write requests to be performed in a single opeartion).
When Deployed in After-Graceful-Shutdown Mode: (Cache data survives only if Cache is gracefully shutdown.)
Single user/thread access: – Superior performance even during single process access.
250% to 300% faster than nearest competitor when performing large numbers of individual read and write operations by a single user.
500% to 600% faster than nearest competitor when performing bulk read and write operations by a single user (where a user submits many read or write requests to be performed in a single opeartion).
Multiple user access or when cache is accessed by many concurrent threads:
Many multiple times faster^* than nearest competitor when performing large numbers of individual read and write operations by multiple threads or users.
An even higher number of multiples faster^* than nearest competitor when performing bulk read and write operations by multiple threads and/or users (where a user/thread submits many read or write requests to be performed in a single opeartion).

^*The TSCache was specifically designed to handle massive concurrent write operations with little or no conflict. The greater the number of concurrent threads and/or users, the greater the performance differential between our cache and our nearest competitors.

Migration

Seamless migration from existing competitors’ cache to TSCache using Dictionary (Key-Value Pair) Interface for Key-Value Pairs.

Enables existing code to work as-is with minimal impact to code.

Enables new code to use more efficient ix interfaces within the same instance of the TSCache (enabling gradual migration if desired).