ZarahDB - JSON based flat file database

The relational database management system is the de facto choice for any standard application to persist data on the disk. It provides all interfaces one requires to store, optimize, and query data for the client. But sometimes persisting data requirement for the application is so basic that we don’t need whole RDBMS engine on the backend to store data - storing in some form of flat file is sufficient enough to serve application data persistence need.

Creating such a backend layer to store data as flat files and querying data from these files can be a bit tedious which can become a project in itself. But there are many open source wrappers already around which can help you to kickstart building such data access layer. ZarahDB is one such open-source project.

ZarahDB is a flat file database engine which uses the combination of the root folder (per database), child folders (per table) and JSON files (per record) to persist data. Storing data as JSON files makes ZarahDB more structured and reliable compare to others. With that, storing data as files make it easy to backup or restore as it become just a matter of copying and pasting application data folder. As with other flat file based database platforms, you don’t need to install any database software on your server.

Project URL: http://zarahdb.com/
NuGet URL: https://www.nuget.org/packages/ZarahDB.Library
Source Code: https://github.com/MikeReedKS/ZarahDB

Using ZarahDB library in your project is easy - Simply add ZarahDB.Library through NuGet and your are ready.

ZarahDB - Nuget

ZarahDB - Nuget

Enough talk, let’s create a very basic .NET Core console application which will store and retrieve an employee information.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
using System;
using ZarahDB_Library;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            var instance = new Uri(AppDomain.CurrentDomain.BaseDirectory + "Data");
            var table = "Employees";

            // Save an employee to the database
            var key = "0001";
            ZarahDB.Put(instance, table, key, "Name", "Mike Smith");
            ZarahDB.Put(instance, table, key, "City", "Chicago");

            // Retrive employee from the database
            var name = ZarahDB.Get(instance, table, key, "Name").Value;
            var city = ZarahDB.Get(instance, table, key, "City").Value;

            Console.WriteLine("Name: " + name);
            Console.WriteLine("City: " + city);

            Console.ReadKey();
        }
    }
}

JSON stored in the Employees data folder (Location: Data\Employees\0\0\0\1\0001.json) will look like following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
{
  "Keys": [
    {
      "Key": "0001",
      "ColumnValues": [
        {
          "Column": "Name",
          "Value": "Mike Smith",
          "PreviousValue": "",
          "Updated": "636935583251802007"
        },
        {
          "Column": "City",
          "Value": "Chicago",
          "PreviousValue": "",
          "Updated": "636935583256218064"
        }
      ]
    }
  ]
}
ZarahDB - Output

ZarahDB - Output

As you probably noticed, ZarahDB stores data values as string, which means you will require to explicitely convert non-string data type or object type as string before passing it to ZarahDB method to save it. This goes other way around as well: you will retrive stored data as string from ZarahDB, you will then require to convert string value to column data type.

Here is the example which explains how DateTime data value to stored and retrived in ZarahDB.

1
2
3
4
5
// Convert non-string data type value to string
ZarahDB.Put(instance, table, key, "BirthDate", (new DateTime(1998, 09, 12)).ToString());

// Convert string to column specifc data type
var birthDate = Convert.ToDateTime(ZarahDB.Get(instance, table, key, "BirthDate").Value);

To store complex object using ZarahDB, you will require to serialize it to Json or Xml string. And as you guessed, you will then require to deserialize returned Json or Xml string to object type.

To conclude this brief overview of ZarahDB, following is the code snippet which demonstrates taking a complex object like a blog post and store and retrive using ZarahDB through Json serialization.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
using Newtonsoft.Json;
using System;
using ZarahDB_Library;

namespace BlogEngine
{
    internal class Post
    {
        public string Title { get; set; }
        public string Content { get; set; }
        public string Author { get; set; }
        public string Category { get; set; }
        public DateTime Publish { get; set; }
    }

    class Program
    {
        static void Main(string[] args)
        {
            var instance = new Uri(AppDomain.CurrentDomain.BaseDirectory + "Data");
            var table = "Posts";

            string key = "quick-start-tutorial";
            var newPost = new Post
            {
                Title = "Quick Start Tutorial",
                Content = "The basic concept of a scattered database is ...",
                Author = "Mike Reed",
                Category = "Database",
                Publish = DateTime.Now
            };
            var jsonPost = JsonConvert.SerializeObject(newPost);

            // Save blog post to the database
            ZarahDB.Put(instance, table, key, "PostData", jsonPost);

            // Retrive blog post from the database
            var resultJson = ZarahDB.Get(instance, table, key, "PostData").Value;
            var resultPost = JsonConvert.DeserializeObject<Post>(resultJson);

            Console.WriteLine("Title: " + resultPost.Title);
            Console.WriteLine("Content: " + resultPost.Content);
            Console.WriteLine("Author: " + resultPost.Author);
            Console.WriteLine("Category: " + resultPost.Category);
            Console.WriteLine("Publish: " + resultPost.Publish.ToString("MM/dd/yyyy"));

            Console.ReadKey();
        }
    }
}
ZarahDB - Blog Example

ZarahDB - Blog Example

As I mentioned in the early part of this article, ZarahDB is great if you have a basic data persistence requirement but surely it’s not recommended for the more complex or data-heavy requirement. I hope you will find this short overview helpful.

Making WCF service request using Postman

You don’t need another WCF client to test WCF services if you already have fantastic Postman installed on your machine. But requesting WCF service using Postman needs little more than just providing a Service Url and a Xml body to the SOAP request. One issue developers always stumble upon first time when using Postman to call a Wcf service is this cryptic error message: The message with Action '' cannot be processed at the receiver, due to a ContractFilter mismatch at the EndpointDispatcher..

Following is an example ActionNotSupported fault response raised by the Wcf Service:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Body>
  <s:Fault>
    <faultcode xmlns:a="http://schemas.microsoft.com/ws/2005/05/addressing/none">
      a:ActionNotSupported
    </faultcode>
    <faultstring xml:lang="en-US">
      The message with Action '' cannot be processed at the receiver, due to a ContractFilter mismatch at the
      EndpointDispatcher. This may be because of either a contract mismatch (mismatched Actions between sender and
      receiver) or a binding/security mismatch between the sender and the receiver. Check that sender and receiver
      have the same contract and the same binding (including security requirements, e.g. Message, Transport, None).
    </faultstring>
  </s:Fault>
  </s:Body>
</s:Envelope>
ActionNotSupported Soap Fault Code raised by Wcf Service

Inspecting the exception fault string more carefully, you will surely realize that the exception has something to do with Action “. Postman is apparently not sending any Action contract with the SOAP request, and Wcf service is treating it as empty Action. With the empty received Action, service trying to match it with its own Action contract which result as mismatch. And this is what is causing the exception.

To resolve the issue, you need to explicitly provide a SOAPAction to the header with your SOAP request to the service. You can get the SOAPAction of the web method from the WSDL document of your Wcf service. Just navigate to the WSDL Uri, which usually your Wcf service Url appended with “?wsdl”, and look for soap:operation node of the web method you want to call. Copy the value of soapAction attribute which will be something like http://tempuri.org/I<Your Service Name>/<Method Name> and use this value as SOAPAction of your service call.

Get the SOAPAction value from WSDL document of your Wcf Service

Adding SOAPAction header in Postman is simple. You have to goto Headers tab of the Postman for the Wcf service, add a New key, provide Key as “SOAPAction” and the Value whatever you copied in above step.

Configure SOAPAction

With the SOAPAction configured for your Wcf service, hit Send button in Postman to make service call and voila! You should receive proper SOAP response from the service. Save the Service request in the Postman so that you don’t have to do the same SOAPAction configuration again for the service.

Wcf service call using Postman

My automatized workflow to setup Postman to call Wcf service is following:

  1. Make GET call of the service Url with appended “?wsdl”.
  2. Copy the value of soapAction attribute of your SOAP method.
  3. Change Http method to POST and remove “?wsdl”
  4. Provide header with key “SOAPAction” and value copied soapAction attribute.
  5. Set “Content-Type” header as “text/xml”
  6. Provide SOAP Envelop in Body
  7. Hit Send (and Save for the future task).

How to go back to previous directory in DOS command prompt?

If you shuffle a lot between Linux/Mac and Windows OS, you surely miss many Linux shell commands in Windows Command Prompt and you definitely wish if you can replicate or have some kind of workaround of those commands in Windows to make life bit easier. One of those commands is navigating back to the previous directory. In Linux, you can just type: cd - and voila, you go back to your previous directory. No need to check paths of previously entered directories. That’s one handy command. As usually happen with most productive shell commands, there is no equivalent command parameter in DOS.

I was looking for some kind of workaround to achieve same in DOS but when I couldn’t find any solution online so I decided to devise and build my own solution. It started with a very basic DOS batch file which just simulates Linux cd – command but slowly and gradually it grows over the period of time as I added more and more features.

So how does it work? To implement this solution, you will need two batch files: ccd.bat and dos_init.cmd. ccd.bat will have actual batch commands which take care of redirecting command to appropriate branch and keeping the history of visited directories whereas dos_init.cmd will create DOSKEY aliases for cd command. You should create both files in one of your PATH directories (for example C:\Windows) so that DOS will able to locate these in Command Windows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
@echo off
if '%*'=='' goto show_current_directory
if '%*'=='~' goto home_directory
if '%*'=='-d' goto my_documents_directory
if '%*'=='-p' goto program_files_directory
if '%*'=='-w' goto windows_directory
if '%*'=='--' goto previous_directory
if '%*'=='-' goto toggle_directory
goto change_directory

:show_current_directory
    cd
    goto batch_end

:home_directory
    call:GotoDirectory %HOMEPATH%
    goto batch_end

:my_documents_directory
    call:GotoDirectory %HOMEPATH%\Documents
    goto batch_end

:program_files_directory
    call:GotoDirectory "%PROGRAMFILES%"
    goto batch_end

:windows_directory
    call:GotoDirectory %WINDIR%
    goto batch_end

:previous_directory
    set current_dir=%cd%
    popd
    goto batch_end

:toggle_directory
    set current_dir=%cd%
    cd %previous_dir%
    set previous_dir=%current_dir%
    goto batch_end

:change_directory
    call:GotoDirectory %*
    goto batch_end

:GotoDirectory
    pushd .
    set previous_dir=%cd%
    cd %~1
    goto:eof

:batch_end
1
2
3
4
5
6
@echo off
doskey cd=ccd $*
doskey cd..=ccd ..
doskey cd--=ccd --
doskey cd-=ccd -
doskey cd\=ccd

You always require running dos_init.cmd before using ccd.bat so that assign aliases are available for ccd batch file. To automate that, you can simply add an entry in Windows Registry which will automatically run dos_init.cmd command every time you open Command Windows.

1
2
[HKEY_CURRENT_USER\Software\Microsoft\Command Processor]
AutoRun=C:\Windows\dos_init.cmd

Following are steps to create above Registry entry. Be careful when editing the registry and make sure you know what you’re doing.

  1. Open Windows Registry.
  2. Go to registry path: [HKEY_CURRENT_USER\Software\Microsoft\Command Processor]
  3. Create a registry entry with key as “AutoRun” and value as complete path of your dos_init.cmd

And that’s all we need.

Let’s go through the usage of this batch file:

Type cd- to toggle between last and current directory.
Type cd-- to go back to previous directories.
Few more examples to changing directory to system folders using this solution.

I hope you will find this hack useful. So how you want to further enhance this batch file? Please do share your changes and improvements here so that others can also benefit.

Stop flickering between pages in IE

If you hate that slight flicker in IE browser between page transitions then you will find this snippet useful. When the user navigates from one page to another, there is a slight white out before next page renders on IE. On other browsers like FireFox or Chrome has a relatively smoother handle for such transition. If your web application has the significant number of post backs then this becomes more annoying to users.

To reduce this flicker in IE, simply add below META tags in the HEAD section of your page.

1
2
<META http-equiv="Page-Enter" content="blendTrans(Duration=0.05)" />
<META http-equiv="Page-Exit" content="blendTrans(Duration=0.05)" />

By reducing the duration of fading content from the previous page and revealing new content from next page, user experience relatively smooth transition between pages. The good thing about above implementation is that above META tags will not have any impact on other browsers.

Getting list of modified objects in SQL Server

Sometimes it always helps to get the list of all modified tables or stored procedures before check-in the code or wrapping up the development for the build. Some developers usually keep this kind list in a file and they update this file every time they modify any table or stored procedure in their development environment. Following are some handy SQL snippets which can provide you list of the tables, or stored procedure modified in the specified duration.

1
2
3
4
5
-- tables modified today
SELECT [name],create_date,modify_date
FROM sys.objects
WHERE modify_date>DATEADD(day,-1,GETDATE())
AND type='U'
1
2
3
4
5
-- stored procedures modified today
SELECT [name],create_date,modify_date
FROM sys.objects
WHERE modify_date>DATEADD(day,-1,GETDATE())
AND type='P'

You can further tweak above query to specify a more specific duration, for example, objects modified in last 7 days.

1
2
3
4
5
-- tables modified in last 7 days
SELECT [name],create_date,modify_date
FROM sys.objects
WHERE modify_date>DATEADD(day,-7,GETDATE())
AND type='U'