JSON Flat File Data Store

https://ttu.github.io/json-flatfile-datastore/
Admin
JSON Flat File Data Store is a lightweight, JSON-based data storage solution for .NET, ideal for...
Tokens:8,046
Snippets:84
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install JsonFlatFileDataStore via .NET CLI and Package Manager

Source: https://ttu.github.io/json-flatfile-datastore

Install the latest version from NuGet using either the .NET Core CLI or the Package Manager Console.

```bash
# .NET Core CLI
$ dotnet add package JsonFlatFileDataStore

# Package Manager Console
PM> Install-Package JsonFlatFileDataStore
```

--------------------------------

### Example user collection in JSON

Source: https://ttu.github.io/json-flatfile-datastore

Shows a sample JSON collection named 'user' with two user objects, each having id, name, age, and city properties.

```json
{
  "user": [
    { "id": 1, "name": "Phil", "age": 40, "city": "NY" },
    { "id": 2, "name": "Larry", "age": 37, "city": "London" }
  ]
}
```

--------------------------------

### Single Item JSON Example

Source: https://ttu.github.io/json-flatfile-datastore

Example of a single item stored in the data store, showing a JSON object with various data types.

```json
{
  "selected_user": { "id": 1, "name": "Phil", "age": 40, "city": "NY" },
  "temperature": 23.45,
  "temperatues": [ 12.4, 12.42, 12.38 ],
  "note": "this is a test"
}
```

--------------------------------

### InsertOne and InsertOneAsync examples

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to insert a single item using InsertOneAsync with dynamic data and InsertOne with typed data. The methods return true if successful. Comments show before/after states.

```csharp
// Asynchronous method and dynamic data
// Before update : { }
// After update  : { "id": 3, "name": "Raymond", "age": 32, "city" = "NY" }
await collection.InsertOneAsync(new { id = 3, name = "Raymond", age = 32, city = "NY" });

// Dynamic item can also be JSON object
var user = JToken.Parse("{ 'id': 3, 'name': 'Raymond', 'age': 32, 'city': 'NY' }");
await collection.InsertOneAsync(user);

// Synchronous method and typed data
// Before update : { }
// After update  : { "id": 3, "name": "Raymond", "age": 32, "city" = "NY" }
collection.InsertOne(new User { Id = 3, Name = "Raymond", Age = 32, City = "NY" });
```

--------------------------------

### Create patch ExpandoObject from Dictionary

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to create a patch ExpandoObject at runtime by converting a Dictionary to JSON and then deserializing to ExpandoObject. The example updates a user's Age, Name, and Work.Name.

```csharp
var user = new User
{
    Id = 12,
    Name = "Timmy",
    Age = 30,
    Work = new WorkPlace { Name = "EMACS" }
};

// JSON: { "Age": 41, "Name": "James", "Work": { "Name": "ACME" } }
// Anonymous type: new { Age = 41, Name = "James", Work = new { Name = "ACME" } };
var patchData = new Dictionary<string, object>();
patchData.Add("Age", 41);
patchData.Add("Name", "James");
patchData.Add("Work", new Dictionary<string, object> { { "Name", "ACME" } });

var jobject = JObject.FromObject(patchData);
dynamic patchExpando = JsonConvert.DeserializeObject<ExpandoObject>(jobject.ToString());

await collection.UpdateOneAsync(e => e.Id == 12, patchExpando);
```

--------------------------------

### InsertMany with typed data

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to insert a list of items using InsertMany. The example inserts two User objects.

```csharp
var newItems = new[]
{
    new User { Id = 3, Name = "Raymond", Age = 32, City = "NY" },
    new User { Id = 4, Name = "Ted", Age = 43, City = "NY" }
};

collection.InsertMany(newItems);
```

--------------------------------

### DataStore and collection lifecycle example

Source: https://ttu.github.io/json-flatfile-datastore

Shows how DataStore loads data from file, collections lazily reference data, and writes are queued. The second collection sees the first insert because data is serialized on first use, but the first collection does not see the second insert.

```csharp
// Data is loaded from the file
var store = new DataStore(newFilePath);

// Lazy reference to the data is created
var collection1st = store.GetCollection("hello");
var collection2nd = store.GetCollection("hello");

// Data is loaded from the store to the collection and new item is inserted
collection1st.InsertOne(new { id = "hello" });

// Data is loaded from the store to the collection and new item is inserted
// This collection will also have item with id: hello as data is serialized when it is used for the first time
collection2nd.InsertOne(new { id = "hello2" });

// collection1st won't have item with id hello2
```

--------------------------------

### UpdateManyAsync with predicate

Source: https://ttu.github.io/json-flatfile-datastore

Updates all items that match the filter. The example updates all items with Age == 30 to age 31.

```csharp
await collection.UpdateManyAsync(e => e.Age == 30, new { age = 31 });
```

--------------------------------

### Full-text search with Find method

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to perform full-text search using the Find method. By default, the search is not case-sensitive. The example searches for 'Alabama' and also demonstrates a case-sensitive search.

```csharp
var store = new DataStore(pathToJson);

var collection = store.GetCollection("user");

// Find all users that contain text Alabama in any of property values
var matches = collection.Find("Alabama");

// Perform case sensitive search
var caseSensitiveMatches = collection.Find("Alabama", true);
```

--------------------------------

### ReplaceItem - Replace existing item

Source: https://ttu.github.io/json-flatfile-datastore

Replaces the item with the given key. Returns true if the item is found. Examples show value and reference types.

```csharp
// Value type
var result = await store.ReplaceItemAsync("counter", 4);
// Reference type
var result = await store.ReplaceItemAsync("myUser", new User { Id = 2, Name = "James" });
```

--------------------------------

### GetNextIdValue - Get next ID value for incrementing fields

Source: https://ttu.github.io/json-flatfile-datastore

Returns the next ID value for incrementing ID fields. For integer IDs, increments last value by one; for strings, parses and increments the number at the end. Supported types: integer and string; Guid not supported as key.

```csharp
var store = new DataStore(newFilePath, keyProperty: "myId");

// myId is an integer
collection.InsertOne(new { myId = 2 });
// nextId = 3
var nextId = collection.GetNextIdValue();

// myId is a string
collection.InsertOne(new { myId = "hello" });
// nextId = "hello0"
var nextId = collection.GetNextIdValue();

collection.InsertOne(new { myId = "hello3" });
// nextId = "hello4"
var nextId = collection.GetNextIdValue();
```

--------------------------------

### Dynamic Data Example with Anonymous Types, JToken, and Dictionary

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to use dynamic data with anonymous types, JToken, and Dictionary<string, object>. All dynamic data is internally serialized to ExpandoObject. Includes insert, update, and LINQ query operations.

```csharp
// Open the database (create new if file doesn't exist)
var store = new DataStore(pathToJson);

// Get employee collection
var collection = store.GetCollection("employee");

// Create new employee
var employee = new { id = 1, name = "John", age = 46 };

// Create new employee from JSON
var employeeJson = JToken.Parse("{ 'id': 2, 'name': 'Raymond', 'age': 32 }");

// Create new employee from dictionary
var employeeDict = new Dictionary<string, object>
{
    ["id"] = 3,
    ["name"] = "Andy",
    ["age"] = 32
};

// Insert new employee
// Id is updated automatically if object is updatable
await collection.InsertOneAsync(employee);
await collection.InsertOneAsync(employeeJson);
await collection.InsertOneAsync(employeeDict);

// Update data from anonymous type
var updateData = new { name = "John Doe" };

// Update data from JSON
var updateJson = JToken.Parse("{ 'name': 'Raymond Doe' }");

// Update data from dictionary
var updateDict = new Dictionary<string, object> { ["name"] = "Andy Doe" };

await collection.UpdateOneAsync(e => e.id == 1, updateData);
await collection.UpdateOneAsync(e => e.id == 2, updateJson);
await collection.UpdateOneAsync(e => e.id == 3, updateDict);

// Use LINQ to query items
var results = collection.AsQueryable().Where(x => x.age > 30);
```

--------------------------------

### Typed Data Example with DataStore and Employee Model

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to use a typed collection with a model class, including insert, update, query, and single item operations. Note that typed collections drop properties not in the model on write; avoid mixing typed and dynamic access.

```csharp
// Define a simple model class
public class Employee
{
    public int Id { get; set; }
    public string Name { get; set; }
    public int Age { get; set; }
}

// Initialize the data store with a JSON file path (creates a new file if one doesn’t exist)
var store = new DataStore("data.json");

// Get a strongly-typed collection
var collection = store.GetCollection<Employee>();

// Create a new employee instance
var employee = new Employee { Id = 1, Name = "John", Age = 46 };

// Insert a new employee
// The id is automatically updated to correct next value
await collection.InsertOneAsync(employee);

// Update employee information
employee.Name = "John Doe";
await collection.UpdateOneAsync(employee.Id, employee);

// Query using LINQ
var results = collection.AsQueryable().Where(e => e.Age > 30);

// Save instance as a single item
await store.InsertItemAsync("selected_employee", employee);

// Single items can be of any type
await store.InsertItemAsync("counter", 1);
var counter = await store.GetItem<int>("counter");
```

--------------------------------

### Update dictionary content with ExpandoObject

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to update a dictionary property using an ExpandoObject. The entire dictionary content is replaced with the update data. The example updates the Scores dictionary of a player.

```csharp
var player = new Player
{
    Id = 423,
    Scores = new Dictionary<string, int>
    {
        { "Blue Max", 1256 },
        { "Pacman", 3221 }
    },
};

var patchData = new ExpandoObject();
var items = patchData as IDictionary<string, object>;
items.Add("Scores", new Dictionary<string, string> { { "Blue Max", 1345 }, { "Outrun", 1234 }, { "Pacman", 3221 }, });

await collection.UpdateOneAsync(e => e.Id == 423, patchData);
```

--------------------------------

### ReplaceItem with upsert option

Source: https://ttu.github.io/json-flatfile-datastore

ReplaceItem with upsert option inserts a new item if the item does not exist. Examples show value and reference types.

```csharp
// Value type
var result = await store.ReplaceItemAsync("counter", 4, true);
// Reference type
var result = await store.ReplaceItemAsync("myUser", new User { Id = 2, Name = "James" }, true);
```

--------------------------------

### ReplaceOne with upsert option

Source: https://ttu.github.io/json-flatfile-datastore

Shows the upsert option for ReplaceOne. If the item does not exist, a new item is inserted with the provided ID. The example inserts a new item with id 11.

```csharp
// New item will be inserted with id 11
collection.ReplaceOne(11, new { id = 11, name = "Theodor" }, true);
```

--------------------------------

### InsertItem - Insert new single item

Source: https://ttu.github.io/json-flatfile-datastore

Inserts a new item into the JSON. Returns true if insertion is successful. Examples show value and reference types.

```csharp
// Value type
var result = await store.InsertItemAsync("counter", 2);
// Reference type
var user = new User { Id = 12, Name = "Teddy" }
var result = await store.InsertItemAsync<User>("myUser", user);
```

--------------------------------

### DeleteMany and DeleteManyAsync - Delete all matching items

Source: https://ttu.github.io/json-flatfile-datastore

Deletes all items that match the filter. Returns true if any items are found. Examples show dynamic and typed usage.

```csharp
// Dynamic
await collection.DeleteManyAsync(e => e.city == "NY");

// Typed
await collection.DeleteManyAsync(e => e.City == "NY");
```

--------------------------------

### ReplaceOne and ReplaceOneAsync with ID and predicate

Source: https://ttu.github.io/json-flatfile-datastore

Replaces the first item matching the ID or predicate. The method returns true if items are found. The example shows sync and async usage with anonymous types and typed User objects.

```csharp
// Sync and dynamic
// Before update : { "id": 3, "name": "Raymond", "age": 32, "city": "NY" }
// After update  : { "id": 3, "name": "Barry", "age": 42 }
collection.ReplaceOne(3, new { id = 3, name = "Barry", age = 33 });
// or with predicate
collection.ReplaceOne(e => e.id == 3, new { id = 3, name = "Barry", age = 33 });

// Async and typed
// Before update : { "id": 3, "name": "Raymond", "age": 32, "city": "NY" }
// After update  : { "id": 3, "name": "Barry", "age": 42 }
await collection.ReplaceOneAsync(3, new User { Id = 3, Name = "Barry", Age = 33 });
```

--------------------------------

### ReplaceMany with predicate

Source: https://ttu.github.io/json-flatfile-datastore

Replaces all items that match the filter. The example replaces all items with City == "NY" with a new object setting City to "New York".

```csharp
collection.ReplaceMany(e => e.City == "NY", new { City = "New York" });
```

--------------------------------

### UpdateOneAsync with ExpandoObject and predicate

Source: https://ttu.github.io/json-flatfile-datastore

Updates the first item matching the ID or predicate. The update data is a dynamic object (ExpandoObject or anonymous type). The example shows async usage with an ExpandoObject and an anonymous type.

```csharp
// Dynamic
// Before update : { "id": 1, "name": "Barry", "age": 33 }
// After update  : { "id": 1, "name": "Barry", "age": 42 }
dynamic source = new ExpandoObject();
source.age = 42;
await collection.UpdateOneAsync(1, source as object);
// or with predicate
await collection.UpdateOneAsync(e => e.id == 1, source as object);

// Typed
// Before update : { "id": 1, "name": "Phil", "age": 40, "city": "NY" }
// After update  : { "id": 1, "name": "Phil", "age": 42, "city": "NY" }
await collection.UpdateOneAsync(e => e.Name == "Phil", new { age = 42 });
```

--------------------------------

### InsertMany with anonymous types and ID generation

Source: https://ttu.github.io/json-flatfile-datastore

Shows how InsertMany handles anonymous types. If an id field is missing, it is added; if present, its value is used. The last user without an id gets an auto-generated id (69).

```csharp
var newItems = new[]
{
    new { id = 14, name = "Raymond", age = 32, city = "NY" },
    new { id = 68, name = "Ted", age = 43, city = "NY" },
    new { name = "Bud", age = 43, city = "NY" }
};

// Last user will have id 69
collection.InsertMany(newItems);
// Item in newItems collection won't have id property as anonymous types are read only
```

--------------------------------

### Update nested collection with null items

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to update items in a collection and add new items. Null items in the update data are skipped, allowing updates at specific indices. The example inserts a family and then adds a second parent and updates the first parent's age.

```csharp
var family = new Family
{
    Id = 12,
    FamilyName = "Andersen",
    Parents = new List<Parent>
    {
        new Parent {  FirstName = "Jim", Age = 52 }
    },
    Address = new Address { City = "Helsinki" }
};

await collection.InsertOneAsync(family);

// Adds a second parent to the list
await collection.UpdateOneAsync(e => e.Id == 12, new { Parents = new[] { null, new { FirstName = "Sally", age = 41 } } });

// Updates the first parent's age to 42
await collection.UpdateOneAsync(e => e.Id == 12, new { Parents = new[] { new { age = 42 } } });
```

--------------------------------

### DeleteOne and DeleteOneAsync - Remove first matching object

Source: https://ttu.github.io/json-flatfile-datastore

Removes the first object matching the filter or id-value. Returns true if an item is found and deleted. Examples show dynamic and typed usage with both id and lambda filter.

```csharp
// Dynamic
await collection.DeleteOneAsync(3);
await collection.DeleteOneAsync(e => e.id == 3);

// Typed
await collection.DeleteOneAsync(3);
await collection.DeleteOneAsync(e => e.Id == 3);
```

--------------------------------

### Run benchmarks from command line

Source: https://ttu.github.io/json-flatfile-datastore

Command to run the JsonFlatFileDataStore benchmarks using the .NET CLI. Executes the benchmark project in Release configuration.

```bash
$ dotnet run --configuration Release --project JsonFlatFileDataStore.Benchmark\JsonFlatFileDataStore.Benchmark.csproj
```

--------------------------------

### Dispose DataStore with Dispose() and using

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to dispose a DataStore manually with Dispose() or automatically with a using block. Dispose waits for pending writes and stops the background thread.

```csharp
// Call dispose method
var store = new DataStore();
// ...
store.Dispose();

// Call dispose automatically with using
using(var store = new DataStore())
{
    // ...
}
```

--------------------------------

### Reload data manually with Reload method and reloadBeforeGetCollection

Source: https://ttu.github.io/json-flatfile-datastore

Demonstrates manual reload using DataStore.Reload and the reloadBeforeGetCollection constructor parameter. Collections created before a reload do not see new data, while collections created after reload do.

```csharp
// Data is loaded from the file
var store = new DataStore(newFilePath);
var store2 = new DataStore(newFilePath, reloadBeforeGetCollection: true);

var collection1_1 = store.GetCollection("hello");
collection1_1.InsertOne(new { id = "hello" });

// Because of reload collection2_1 will also have item with id: hello
var collection2_1 = store2.GetCollection("hello");

collection2_1.InsertOne(new { id = "hello2" });

store.Reload()

// Because of reload collection1_2 will also have item with id: hello2
var collection1_2 = store.GetCollection("hello");

// collection1_1 will not have item with id: hello2 even after reload, because it was initialized before reload
```

--------------------------------

### AsQueryable LINQ query with dynamic data

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to query a collection using LINQ with dynamic data. The DataStore is initialized with a JSON file path, and the query finds the first user with name 'Phil'.

```csharp
// Initialize the data store with a JSON file path (creates a new file if one doesn’t exist)
var store = new DataStore(pathToJson);

var collection = store.GetCollection("user");

// Que item with name
var userDynamic = collection
                    .AsQueryable()
                    .FirstOrDefault(p => p.name == "Phil");
```

--------------------------------

### Minify JSON output with minifyJson parameter

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to enable minified JSON output by passing minifyJson: true to the DataStore constructor. The default is indented output.

```csharp
var store = new DataStore(newFilePath, minifyJson: true);
```

--------------------------------

### AsQueryable LINQ query with typed data

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to query a collection using LINQ with typed data. The DataStore is initialized with a JSON file path, and the query finds the first user with Name 'Phil'.

```csharp
var store = new DataStore(pathToJson);

var collection = store.GetCollection<User>();

// Find item with name
var userTyped = collection
                    .AsQueryable()
                    .FirstOrDefault(p => p.Name == "Phil");
```

--------------------------------

### Collection naming with GetCollection and GetCollection<T>

Source: https://ttu.github.io/json-flatfile-datastore

Shows how collection names are converted to the selected case. Dynamic collections require a name; typed collections use the class name converted to the selected case unless a name is provided.

```csharp
var store = new DataStore(newFilePath);
// JSON { "movie": [] };
var collection = store.GetCollection("movie");
// JSON { "movie": [] };
var collection = store.GetCollection("Movie");
// JSON { "movie": [] };
var collection = store.GetCollection<Movie>();
// JSON { "movies": [] };
var collection = store.GetCollection<Movie>("movies");
```

--------------------------------

### Write JSON in lower or upper camel case

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to set the JSON casing via the useLowerCamelCase parameter in the DataStore constructor. Default is lower camel case; passing false writes upper camel case.

```csharp
// This will write JSON in lower camel case
// e.g. { "myMovies" : [ { "longName": "xxxxx" } ] }
var store = new DataStore(newFilePath);

// This will write JSON in upper camel case
// e.g. { "MyMovies" : [ { "LongName": "xxxxx" } ] }
var store = new DataStore(newFilePath, false);
```

--------------------------------

### Encrypt JSON file content with encryptionKey

Source: https://ttu.github.io/json-flatfile-datastore

Pass the encryptionKey parameter to the DataStore constructor to encrypt written JSON data using AES-256.

```csharp
var secretKey = "Key used for encryption";
var store = new DataStore(newFilePath, encryptionKey: secretKey);
```

--------------------------------

### GetItem - Retrieve single item

Source: https://ttu.github.io/json-flatfile-datastore

Retrieves a single item by key. For typed data, throws KeyNotFoundException if key not found; for dynamic and nullable types, returns null.

```csharp
var store = new DataStore(pathToJson);
// Typed data
var counter = store.GetItem<int>("counter");
// Dynamic data
var user = store.GetItem("myUser");
```

```csharp
// throw KeyNotFoundException
var counter = store.GetItem<int>("counter_NotFound");
var user = store.GetItem<User>("user_NotFound");
// return null
var counter = store.GetItem<int?>("counter_NotFound");
var counter = store.GetItem("counter_NotFound");
```

--------------------------------

### ID generation for string and missing id fields

Source: https://ttu.github.io/json-flatfile-datastore

Shows how ID generation works for string ids and missing id fields. If the id is a string, an incremented number is appended. If missing, a new id is generated based on the latest id.

```csharp
// Latest id in the collection is "hello5"
var user = JToken.Parse("{ 'id': 'wrongValue', 'name': 'Raymond', 'age': 32, 'city': 'NY' }");
await collection.InsertOneAsync(user);
// After addition: user["id"] == "hello6"

// User data doesn't have an id field
var userNoId = JToken.Parse("{ 'name': 'Raymond', 'age': 32, 'city': 'NY' }");
await collection.InsertOneAsync(userNoId);
// After addition: userNoId["id"] == "hello7"
```

--------------------------------

### C# dynamic lambda CS1977 workaround

Source: https://ttu.github.io/json-flatfile-datastore

Shows how to avoid CS1977 when passing a dynamic object to ReplaceOne. Cast the dynamic to object or to Predicate<dynamic> to satisfy the compiler.

```csharp
dynamic dynamicUser = new { id = 11, name = "Theodor" };

// This will give CS1977 error
collection2.ReplaceOne(e => e.id == 11, dynamicUser);

// Compiler will accept this
collection2.ReplaceOne(e => e.id == 11, dynamicUser as object);

// Compiler will also accept this
collection2.ReplaceOne((Predicate<dynamic>)(e => e.id == 11), dynamicUser);
```

--------------------------------

### DeleteItem - Remove single item

Source: https://ttu.github.io/json-flatfile-datastore

Removes the item with the given key. Returns true if item is found and deleted. Shows sync and async usage.

```csharp
// Sync
var result = store.DeleteItem("counter");
// Async
var result = await store.DeleteItemAsync("counter");
```

--------------------------------

### UpdateItem - Update first matching item

Source: https://ttu.github.io/json-flatfile-datastore

Updates the first item matching the filter with properties from a dynamic object (anonymous type or ExpandoObject). Returns true if item is found.

```csharp
// Value type
var result = await store.UpdateItemAsync("counter", 2);
// Reference type
var result = await store.UpdateItemAsync("myUser", new { name = "Harold" });
```
Tokens:
Raw