Tessellation Software LLC
Enterprise Solution for Scalable and Secure Performance

What is the TSCache?

Like all caches, the TSCache was designed to enable rapid access to a very large set of information as if that information was all available from directly accessible memory (RAM), but without requiring an unduly large amount of RAM in order to operate -- keeping all but the most recently used information on disk/SSD. Also like other caches, this cache enables this information to be accessed through a set of relatively smaller “keys”, hiding the complexity of managing this information and providing access seamlessly.

The TSCache (like some but not all caches) offers concurrent, thread-safe access.

It may be “embedded” with an application executing on a single server or PC; or, it may be deployed in “client-server mode” (where multiple applications executing on other PCs or servers may access a common cache), or in “distributed” mode (whereby a single TSCache is replicated on multiple server instances providing for faster read access by a greater number of users and robustness in the event of server failure [i.e., replication]). It also offers restart capability (at a performance cost) and transactional capability (at a greater performance tradeoff).

How is the TSCache different from its competitors?

The TSCache offers 4 different types of interfaces for accessing data whereas competitors generally offer one. These interfaces are not just different ways of storing and accessing data, but provide for key functionality differences – e.g., multiple keys to a single data item and Big Data capabilities with multi-dimensional search. The different interfaces are discussed below.

The TSCache enables bulk operations that provide high performance when storing and retrieving multiple cached data items in a single call. These operations are often performed in parallel by this Cache -- providing unparalleled (if you'll forgive the pun) performance.

There are tremendous advantages that enable rapid concurrent access by multiple users, processes and/or threads. These advantages provide for extreme performance through the parallelization and concurrency aspects of the design and implementation of this Cache. Importantly: Provides for fair scheduling of resources when there are multiple processes/threads and full allocation of all resources (allocated cores) whether engaged by one or multiple processes.

The TSCache enables in-place modification of very large single data items (e.g., video or genetic sequences) thus obviating the need for retrieving and re-saving these items in their entirety during modifications.

Interfaces

We offer 4 Interfaces:
  1. 1.  Ix (Index) interface
  2. 2.  Key Cache Interface
  3. 3.  Dictionary (Key-Value Pair) Interface
  4. 4.  Tagged Ix Interface

3 of our 4 interfaces are unique. Other caches we are aware of offer only the Dictionary Interface. Some advantages of this cache is faster speed -- especially during concurrent access.

1.   Ix Interface

  • Provides for unlimited array-like access to data;
  • Stores data and returns an integer index; then, enables retrieval of this data (or pieces of this data) using that index.
  • int     allocateAndStore(byte [ ] dataOfAnySz)    ….. store
  • byte [ ]     retrieve(int ix)      ….. retrieve
  • byte [ ]     retrievePartial(int ix, int startPos, int endPos)     ….. retrieve partial
  • void      store(int ix, byte [] dataOfAnySz)      ….. replace
  • void      remove(int ix)      ….. delete (free up space)
  • byte []      retrieveAndRemove(int ix)
Advantages:
  • Provides for faster storage and access than Dictionary and Key Cache Interfaces.
  • The retrievePartial( ) capability is extremely useful when the data stored is very large (e.g., a genetic sequence).

2.  Key-Cache Interface

  • Ability to define Data Types and to retrieve property information from byte arrays stored.
  • Capability to define multiple keys for each datatype and ability to retrieve an item from any of multiple keys.

3.  Dictionary (Value-Key) Interface

  • Key-value pair access -- similar to the manner that almost all commercial and open source caches work today.
  • Provides capability to drop this cache into existing code with minimal requirements for code change.

4.  Ix Tagged Interface

  • A combination of Ix Interface with the ability to tag cached objects with one or more tags.
  • Capability of creating tag hierarchies – a key facilitator for data mining/Big Data analysis.
  • Provides log2-N performance multi-dimensional data access/recall for any combination of Tags.

Deployment

Embedded

  • Thread-safe: Enables multi-core creation of cached data.
  • Concurrent access: Enables multi-core use and analysis of large-scale data local to Purchaser’s code
  • Enables extreme performance in workstation-based applications

Client Server

  • Enables multiple client access to data cached on a single large server.

Distributed (Coming soon)

  • Enables deployment of a single cache executing across a grid of machines.
  • Replication: Provides performance, robust operation (protects against single server failure).
  • Data sharding: Enables data to be distributed among many execution engines – both for performance during insertion/modification and query execution performance.

Robustness

    Transactional (Coming soon [Transactionality for single operations already provided -- see below.])

    • Enables transactionality across Cache Operations – single or multi (replicated) instances.
    • With Replication, guarantees transactionality across instances.

    Always

    • Atomic Cache Operations – A single operation will either be completely done or undone on any single instance.
      • Performance is far higher than "Transactional" due to attention to detail in design of reversible steps if a single operation is not completed.
      • Transactional -- but only for a single cache operation (not across cache operations).

    After Graceful Shutdown

    • As long as Cache is shutdown gracefully, all data will be intact when the cache is brought back up. Otherwise, cache will be brought up empty.
    • More than an order of magnitude faster than modes mentioned above.

    Clean

    • Cache is always brought up as empty and starts fresh whether shutdown gracefully or not.
    • More than an order of magnitude faster modes mentioned above. (Same as "After Graceful Shutdown", so this is a choice.)

Performance

PBEF – Participatory Bulk Execution Framework

  • Bulk Operations – sub-divided and handled by a pool of a configurable number of worker threads
  • Revolutionary Scheduling Mechanism:
  • Enables tasks submitted later to share worker thread resources
  • Two-tier prioritization w/lower-tier still receiving a minimal configurable %age of resources during execution of higher priority tasks.
  • All users receive their fair share of resources regardless of when they submit (so quick operations do not wait for larger operations submitted prior to complete).
  • Participatory – dedicates invoking thread to task submitted
  • Submitting thread participates with some/full bulk worker thread participation to accomplish task.
  • Guarantees performance equal to or better than executing thread alone.
  • Work units within a bulk operation can be batched.
  • Participatory is optional – PBEF can be used w/passed callback for async submission w/all other benefits
  • User receives maximum performance possible given the total workload placed on the TSCache at any given time.

Attention to detail across nodes coupled with unique design elements lead to superior performance

  • In Transaction Restartable Mode: Unique replication design among instances – replicating via positional data vs. repetition of execution via replicated logs. (Coming soon!)
  • In Always Restartable Mode: Detailed reversible processes ensure that data is written to durable storage only at the end of steps where necessary to enable reversibility if atomic operation is not completed; and, importantly, not during other steps.
  • In AfterGracefulShutdown and in "Clean" Mode: Data is never forced to durable storage until the TSCache is logically closed.

Free Block Allocation Mechanism

  • Non-fragmentation model: Guarantees no performance degradation over time.
  • Pre-allocation of DataBlocks: Ensures space for new items can be obtained quickly.

Configuration Options to Optimize Performance – Configurable in accordance with characteristics of

  • Purchaser’s Data: Size and Distribution of elements of data
  • Purchaser's Hardware: Cores (number/speed), Memory and Type of Durable Storage
  • Purchaser's Usage: E.g., whether
  • Largely read-only with occasional writes vs. large turnover [like Market Data]
  • There will be concurrent and/or multi-threaded access vs. single user access, etc.

Ix Interface Performance

  • Faster than dictionary (key-value pair) access of industry-leading competing caches

  • Optimized for multi-user, multi-threaded and concurrent access

  • Peformance gap increases significantly vs. competitors when accessed by a large number of users/threads concurrently
  • Fastest performance for concurrent writes.

  • Big Data

    Multi-Keyed


    • As with a database, ability to have multiple keys to access a single TSCache data item.
    • Index on keys for a data item updated concurrently.

    Tagging

    • Ability to retrieve items tagged with any combination of tags (AND and OR)
    • Multi-dimensional performant search: Provides this capability with unique design enabling recall of items with combinations of tags in an indexed manner (without scanning)
    • Parent hierarchies of tags – enabling recall of all sub-tags of a parent with parent tag

    Competitor Comparison

    Functionality

    • No Competitors – except Berkeley DB (which, isn’t a Cache) support multiple keys on a single item.
    • No Competitors have an Ix Interface (although that can be simulated with a numeric key).
    • No Competitors support tagging and the multi-dimensional search that this provides.
    • No Competitors have the load-balancing capabilities of this Cache when accessed by multiple concurrent threads/users and none prevent resource starvation and provide for fair allocation (vs. a first-come, first-serve approach when multiple processes are retrieving/writing large amount of data)
    • No Competitors provide for modification of a piece of a single large item of data in the cache (e.g., a piece of a video or genetic sequence) without retrieving and re-saving the entire piece of data.

    Performance


    • When Deployed in Always-Restartable Mode:
    • Single user/thread access: – Superior performance even during single process access.
    • 35% to 40% faster than nearest competitor when performing large numbers of individual read and write operations by a single user.
    • 400% to 500% faster than nearest competitor when performing bulk read and write operations by a single user (where a user submits many read or write requests to be performed in a single opeartion).
    • Multiple user access or when cache is accessed by many concurrent threads:
    • ??? faster than nearest competitor when performing large numbers of individual read and write operations by a single user.
    • ??? faster than nearest competitor when performing bulk read and write operations by a single user (where a user submits many read or write requests to be performed in a single opeartion).
    • When Deployed in After-Graceful-Shutdown Mode:
    • Single user/thread access: – Superior performance even during single process access.
    • 250% to 300% faster than nearest competitor when performing large numbers of individual read and write operations by a single user.
    • 500% to 600% faster than nearest competitor when performing bulk read and write operations by a single user (where a user submits many read or write requests to be performed in a single opeartion).
    • Multiple user access or when cache is accessed by many concurrent threads:
    • ??? faster than nearest competitor when performing large numbers of individual read and write operations by a single user.
    • ??? faster than nearest competitor when performing bulk read and write operations by a single user (where a user submits many read or write requests to be performed in a single opeartion).

    Migration

    Seamless migration from existing competitors’ cache to TSCache using Dictionary (Key-Value Pair) Interface for Key-Value Pairs.


  • Enables existing code to work as-is with minimal impact to code.
  • Enables new code to use more efficient ix interfaces within the same instance of the TSCache (enabling gradual migration if desired).