03. Data Persistence - SQL and NoSQL, Memory Stores, GraphDB

POJO and JSON

POJO: Plain Old JS Object. Essentially a struct.

JSON is very similar to a POJO:

• Data-interchange format
• No versioning
• For serializing data

SQL

ACID:

• Atomic: transaction is either fully completed or fully fails
• Consistency: DB is in a valid state before and after the transaction (invariants preserved)
• Isolated: transactions should be isolated from other in-progress transactions
• Durable: once transactions are committed, they should not be lost

CAP Theorem: if you have a distributed system, pick two:

• Consistency: every read receives the most recent data
• Availability: every read receives a response
• Partition tolerance: system can function even if network goes down

BASE: give up consistency and get:

• Basic availability (through replication)
• Soft state: the state can change over time - this happens due to:
• Eventual consistency: the data will be consistent eventually (if the data is not changed frequently)

A BASE transaction may act like multiple transactions in ACID.

Key-Value Stores

Don’t bother defining schemas, just plonk a key and value together.

• Upsides: fast, simple, flexible, scalable
• Downsides:
  • No validation at all
  • Consistency checking offloaded from DB to application
  • No relationships
  • No aggregate operations
  • No search operations (other than via key)

In-memory data store (e.g. redis):

• Stores whole DB in memory
• Useful for caching common responses

Document Database

• Bunch of JSON/XML files indexed and stored in DB
• Lots of duplicated data
• Risk of inconsistent document structures

Graph Database

• Node: entity
• Edges: relationship between entities
  • Can be uni/bi-directional
  • Weights map to relationship properties
• Properties: describe attributes of a node/edge
• Hypergraph: one edge joining multiple nodes

Semantic web:

• Subject-predicate-object (nodes are nouns, edges are verbs, attributes are adjectives/adverbs respectively)
• Nodes are URIs/values
• Represented as Resource Description Framework (RDF) triple stores
• Linked data on the web

Labelled property graph:

• Nodes and edges have an internal structure
• More efficient to store than RDF

Graph databases give measures of centrality such as:

• Degree: indegree/outdegree
• Closeness: average length of all shortest paths
• Betweenness: number of times a node acts as a bridge along the shortest paths
• Eigenvector: influence of a node in the network

Query languages available for graph databases (e.g. Cyper for neo4j).