Tuesday, August 6, 2019

Using EntityFramework in a .Net Core Project

Introduction

While I work with EntityFramework every day, I do not add an EntityFramework (EF) to an existing project every day. Time and again, I have to create a new project or add EntityFramework to the existing project and I have to re-do many steps that are hard to remember. I decided to document the steps in the form of this article.

In this article, I am going to show how to get started with EntityFramework. I will start a project without support to EntityFramework and add the EF support to the project and use a migration to update the database. The code related to this article is available at GitHub. The sample project uses in Visual Studio 2017 IDE using C#, MS SQL Server with SQL Server Management Studio (SSMS) and EntityFrameworkCore with .Net Core SDK 2.2.


Create Solution and Project

I started with the creation of an empty solution and added a WebAPI project targetting .NetCore 2.2.





As I am using MS SQL Server, I added a reference to

Microsoft.EntityFrameworkCore.SqlServer and Microsoft.EntityFrameworkCore.Design using Nuget Package Manager.


I updated the appsetting.json file for the connectionstring by adding the following property:

 "ConnectionStrings": { 
     "DefaultConnection":
        "Server=(localdb)\\mssqllocaldb;Database=ApplicationDb;Trusted_Connection=True;" }



Configure Database and Models

I have created models for Customer, Contact, and CustomerContact. The code example has these models in DBContext folder. In addition, there are classes for IAuidtable and Audit and some enums which can be ignored for now. Customer and Contact entity have many to many relationships and thus I added CustomerContact entity to store the relationship.


public class Customer : IAuditable
    {
        public Guid Id { get; set; }
        public String AccountNumber { get; set; }
        public String Name { get; set; }
        public DateTime? CreatedDate { get; set; }
        public DateTime? ModifiedDate { get; set; }
        public String LastModifiedBy { get; set; }
        public bool IsInactive { get; set; }
        public ICollection<CustomerContact> CustomerContacts { get; set; }
    }

    public class Contact : IAuditable
    {
        public Guid Id { get; set; }
        public String Name { get; set; }
        public String Title { get; set; }
        public String Phone { get; set; }
        public String Email { get; set; }
        public ContactTypeEnum ContactType { get; set; }
        public String Note { get; set; }
        public DateTime? CreatedDate { get; set; }
        public DateTime? ModifiedDate { get; set; }
        public String LastModifiedBy { get; set; }
        public bool IsInactive { get; set; }
        public ICollection<CustomerContact> CustomerContacts { get; set; }

    }

    public class CustomerContact:IAuditable
    {
        public Guid Id { get; set; }
        public DateTime? CreatedDate { get; set; }
        public DateTime? ModifiedDate { get; set; }
        public string LastModifiedBy { get; set; }
        public bool IsInactive { get; set; }
        public Guid CustomerId { get; set; }
        public Customer Customer { get; set; }
        public Guid ContactId { get; set; }
        public Contact Contact { get; set; }
    }

    public class Audit
    {
        public Guid Id { get; set; }
        public Guid? EntityId { get; set; }
        public string User { get; set; }
        public String Entity { get; set; }
        public DateTime DateTime { get; set; }
        public string ColumnName { get; set; }
        public String OldValue { get; set; }
        public String NewValue { get; set; }
        public EntityStateChangeTypeEnum ChangeType { get; set; }

    }
    /// <summary>
    /// This interface determines what will be automatically tracked.
    /// </summary>
    interface IAuditable
    {
        Guid Id { get; set; }
        DateTime? CreatedDate { get; set; }
        DateTime? ModifiedDate { get; set; }
        String LastModifiedBy { get; set; }
        bool IsInactive { get; set; }
    }

    public enum EntityStateChangeTypeEnum
    {
        Added,
        Deleted,
        Modified,
    }

    public enum ContactTypeEnum
    {
        Primary,
        Secondary,
        Emergency,
    }


With these changes, I am ready to set up code-first logic for creating database tables. I am going to create a class ApplicationDbContext that derives from DbContext of EntityFrameworkCore. In this class, I create DbSet objects of the type of Customer, Contact, CustomerContact, and Audit. This is what makes EF aware of the tables I need to create.


using Microsoft.EntityFrameworkCore;

namespace AutotrackEntityChange.DBContext
{
    public class ApplicationDbContext: DbContext
    {
        //Define DbSets
        public DbSet<Customer> Customers { get; set; }
        public DbSet<Contact> Contacts { get; set; }

        public DbSet<CustomerContact> CustomerContacts { get; set; }
        public DbSet<Audit> Audits { get; set; }

        public ApplicationDbContext(
          DbContextOptions<ApplicationDbContext> options)
            : base(options) {}
    }
}


Now I need to add service in the Startup.cs class to use ApplicationDbContext within the application. The service can be added inside the ConfigureService method using the following code:


 services.AddDbContext<ApplicationDbContext>(options =>
        options.UseSqlServer(Configuration
            .GetConnectionString("DefaultConnection")));


Note that the service addition above uses the same DefaultConnection that was defined in the appsetting.json earlier.


Migration and Database Update

I can run the 'Add-Migration 'InitialCreate'' command in the Package Manager Console within the Visual Studio to generate necessary migration scripts.


This will create necessary Migration files in the solution under Migration folder.


Now I can run 'Update-Database' command in the Package Manager Console to apply the migration. 

Note that at this time, the migration code will generate the database and the tables inside 'ApplicationDb' database. I can now open SSMS and navigate to the database and see the tables.



Summary

In this article, I showed steps to add EntityFramework support to a .Net Core project using Code-First approach and migrations. I provided an example using a WebAPI project with SQL Server, but the steps mentioned here can be used for other database and other project types. The migration I used was really for creating the database and entities, but a similar approach can be used to update the entities. am hoping this will help someone and I welcome comments and suggestions.

References

Sunday, August 4, 2019

Remove commit history in git

Have you ever had a need to clean up all the history in the git repository? I have. I dealt with this old fashioned way until I started interacting with the command line.  The old fashioned way is to create a new repository and update files and replace the old with a new one. In this article, I am going to show how to clean the commit history in git using tools and technique using command lines.  

Method 1: Re-init the repository

In this method, I remove and recreate the git and modify the origin and then I can push the code to remote. The following set of commands can be used:
$ rm -rf .git
$ git init
$ git add .
$ git commit -m "Starting Commit"
$ git remote add origin [......]
$ git push -u --force origin master

Method 2: Alter the branch


In this method, I initially create a temp branch, add all the files to that branch, delete the old branch and then rename the current branch to master and then push the changes to remote. Following sets of commands can do the job. 
$ git checkout --orphan temp
$ git add -A
$ git commit -am "Starting Commit"
$ git branch -D master
$ git branch -m master
$ git push -f origin master

Method 3:  Squashing the commits 

In this method, I combine all the commits into one which is known as squashing. For example, if I have 25 commits since the beginning, then I can combine all commits into one by using the following steps for squashing commits into one and amending the commit message:
$ git rebase -i Head~25
$ #edit the commits mannually
$ git commit –amend

Wednesday, July 31, 2019

Unit Testing C++ Code with Visual Studio

Unit Testing has been an inherent part of software development today. In Unit Testing, individual units of software are tested. While manual testing of individual functionality has been part of the development cycle, more recently these testing is automated with a testing framework.

In this article, I am going to illustrate the unit testing example in C++ using Visual Studio 2017 IDE and Native Unit Test Project. For illustration, unit test and actual software are contained within a solution. The code is contained in the basiccpp solution which currently includes one project and one test project and can be obtained from GitHub.

To get started, I have a solution named ‘basiccpp’ solution. The solution currently contains a console application, basiccpp, written in C++.

Step 1: Add a test project.
In the solution explorer, right-click to get the context menu and click ‘Add’ and select Test. Select the test framework. In this example, I am using ‘Native Unit Test Project’. The native unit test project resides in namespace “Microsoft::VisualStudio::CppUnitTestFramework”.


The test project created will test the functions within the basiccpp project. The basiccpp project is a window console application written in C++. 


Step 2: Add a reference

For the test project to get access to the basiccpp project, I will add a reference to the project.
Right-click the ‘References’ in the test project, and click Add Reference and in the window displayed, check the basiccpp project. If a single test project needs to test multiple projects, then I would select all the projects I am testing. 


Now the reference is added, I can start creating tests for the projects.

Step 3: Add Test
By default, Visual Studio template provides us with a unittest1.cpp file. I am going to rename this such that I know by looking at the file name the behavior I am testing. In this example, I am going to test Array.cpp file. Thus, I would name it as ‘ArrayTest.cpp’.

The specification of Array is in Array.hpp file and actual impelementaiton is in Array.cpp file. The content of this Array.hpp file is shown below:

#pragma once
#ifndef ARRAY_HPP_INCLUDED
#define ARRAY_HPP_INCLUDED
#endif

#include <cassert>
#include <ostream>
#include <utility>

template <typename T>
class Array {
private:
       T* m_ptr{ nullptr };
       int m_size{ 0 };
public:
       Array() = default;
       ~Array();
       //Constructor to create an array of defined size
       explicit Array(int size);
       //copy constructor
       Array(const Array& source);
       T& operator[](int index);
       T operator[](int index) const;
       void print() const;
       int Size() const;
       bool IsValidIndex(int index) const;
};

For testing, I will add test targeting publicly exposed the function of Array<T> class.
Since I am targeting functions defined in Array.hpp in basiccpp project, I am going to include Array.cpp in the ArrayTest.cpp by adding the following line:

#include "..\basiccpp\Array.cpp"

The Visual Studio will provide the IntelliSense as I am typing to the referenced project and I can select the correct include file.  




Step 4: Create tests

I am going to update the content of ‘ArrayTest.cpp’. In minimum, the structure of the test class one method with the following structure: 

namespace basicTest
{
       TEST_CLASS(my_test_class)
       {
       public:

              //test creation, size and [] operators.
              TEST_METHOD(my_test_method)
              {
                      //TODO: add test logic here.
              }
}
}

I can create as many methods within the class. The full content of the file test class for ArrayTest is shown below:

#include "stdafx.h"
#include "CppUnitTest.h"
#include "..\basiccpp\Array.cpp"
#include <functional>
#include <assert.h>

using namespace Microsoft::VisualStudio::CppUnitTestFramework;

namespace basicTest
{
       TEST_CLASS(ArrayTest)
       {
       public:

              //test creation, size and [] operators.
              TEST_METHOD(ArrayTest_Size)
              {
                     int size = 5;
                     Array<int> mArray{ size };
                     for (int i = 0; i < size; i++) {
                           mArray[i] = i;
                     }
                     Assert::AreEqual(size, mArray.Size());
                     Assert::IsTrue(mArray[0] == 0);
                     Assert::IsTrue(mArray[4] == 4);
              }

              //test the empty function
              TEST_METHOD(ArrayTest_Empty)
              {
                     int size = 5;
                     Array<int> mArray0{ 0 };
                     Assert::IsTrue(mArray0.IsEmpty());

                     Array<int> mArray{ size };
                     for (int i = 0; i < size; i++) {
                           mArray[i] = i;
                     }
                     Assert::IsFalse(mArray.IsEmpty());
              }

              //test  isValidIndex function
              TEST_METHOD(ArrayTest_IsValidIndex)
              {
                     int size = 5;
                     Array<int> mArray{ size };
                     Assert::AreEqual(size, mArray.Size());
                     Assert::IsTrue(mArray.IsValidIndex(1));
                     Assert::IsFalse(mArray.IsValidIndex(5));
              };

              //Test the method indexoutofbount by passing the invalid index and checking
              //if the exception is thrown
              TEST_METHOD(ArrayTest_IndexoutOfBound)
              {
                     int size = 5;
                     Array<int> mArray{ size };
                     auto func = [mArray, size] {
                           mArray[size + 100]; };
                     Assert::ExpectException<IndexOutOfBoundsException>(func);
              }

              //test copy constructor
              TEST_METHOD(ArrayTest_CopyConstructor)
              {
                     int size = 5;
                     Array<int> mArray{ size };
                     for (int i = 0; i < size; i++) {
                           mArray[i] = i;
                     }
                     Array<int> cArray = mArray;
                     cArray[0] = 100;
                     Assert::IsTrue(cArray[0] != mArray[0]);
              }
       };
}

Step 5: Run the test using Test Explorer
Once I have written the test, I can build and run the test using Test Explorer. In addition, by adding breakpoints to the method as well as in the test, I can debug the test. 


Additional Reference:
Write unit tests for C/C++ in Visual Studio
Using Assert::ExpectException with Native Unit Testing in VS11
GitHub page for the code
GitHub Link for Code

Friday, June 14, 2019

Combine multiple git commits into a single commit in GIT using squash



Introduction

As a developer, one wants to keep the commits cleaner. At the same time, while developing I also want to keep the detailed record of changes I made and commits I make. From the start to the end, depending on scope, I might have made many trivial commits and after I am happy with the changes, I would like a few (sometimes one while at other times multiple) commits to indicate what changed, rather than to have several commit messages.

Squash in GIT can be used to combine multiple individual commits into one. This makes commits in the repository cleaner. For illustration, let’s say that we have about 25 commits from the beginning through the end of development on an issue. During those 25 commits, several code changes were made which may not be of high importance to remain in the repository. Those 25 commits can be wrapped to a single commit before it can be merged into a master branch.

The rebase command in the interactive mode (i.e. -i or –interactive) can be used to combine those comments into one.

I am using git bash in this article and I have 25 commits on a branch which I want to modify to have just one commit reflecting a feature has been completed.

Git commit history


Rebasing

Run the following command to ensure that I am picking 25 commits in the branch I am in.
git rebase -i Head ~25
Git Bash Window

The above command is instructed to get 25 commits from the head. Upon execution, the git opens a text editor showing the commit details (messages) of the last 25 commits.  

Editor to mark commits as squash or pick

I can edit the commits by putting ‘Squash’ on all the commits by changing ‘pick’ to ‘squash’ and leaving one commit with ‘pick’, probably the first one in the list i.e. the latest. We can change that message later on. In the example, I am picking the first one as shown in the figure: 

Editor after marking the commits as squash or pick


Save the changes in the editor by clicking the save button on the top left and close by the window by clicking ‘x’ on top right.

After rebasing in completed, the branch will change to show the commit message what had ‘pick’ in the editor.

Updating Commit Message
The commit message may not be relevant at this point and thus one may want to change the commit message. This can be achieved by
git commit –amend
The above command will open an editor just like before showing all the previous commit messages that can be changed to relevant message and save and close the editor. After which git would have updated the message.

Handling Remote
If the branch was previously pushed to remote, we need to synchronize the commits to remote as well.

Since the branch appears different in remote it might find conflicts. Thus, we must force the local branch which will replace the remote with the latest local changes by squash.
Git push –force

With the rebase, amend and force I was able to combine the twenty-five smaller commits into one with one clear message, both in remote and local.

Monday, June 10, 2019

Interacting with Microsoft Access Database in .Net Core Application

I had to work with a program to read/write Microsoft Access Database and I was using .Net Core 2.2 application for processing a large number of files. I later found out that Oledb is no longer supported in the .Net Core world, and then I had to explore additional options. While the Entity Framework option is there, but it would be overkill for the programming needs for the project. I looked into the ODBC based approach and it showed the promise.  After some issues and errors, I was able to successfully connect/interact with the access db.

The AccessDatabaseEngine_X64.exe needed to be installed in the machine. The error message that is thrown due to the missing correct driver (I had 32 bit version before) was vague so it was not immediately evident, but eventually, it worked out.

In this article, I am going to discuss the steps to take in order to connect to access db from within a .Net Core 2.2 application written in C# using Visual Studio 2017.

The content of app.config is fairly simple as shown below which stores a key value for datapath:

The following code snippet shows a method named 'ProcessFile(..)' demonstrating demonstrates the use of interaction with Access. The method processes the *.accdb file in the folder path, an input to the method itself. The method looks into the folder path for an access db file. If the file is found, a connection is made and names of the tables (user tables only) are written in the console. In case of error, it writes the error message.

Wednesday, April 17, 2019

A simple branching strategy to work in CI/CD and Code Review Policy in Agile

Code review is an important step in DevOps to catch any error before the code goes into the production. In a Continuous Integration/Continuous Delivery (CI/CD) model, code review is even important. Tools exist to enforce code review policy in place at source. The idea is that any code change must first be reviewed before the changes are merged to the master branch with the assumption that the master branch only contains duly reviewed code to ensure quality. This article assumes the use of GIT for source control and Team Foundation Server (TFS) or Azure DevOps for build/release and CI/CD. 

When working as a developer, one wants to test the code before it can be reviewed. For testing, in most cases, the build and release pipelines are configured to listen to changes in the master branch. In such cases, a code review policy can overburden the developers by requiring to review a code that may not work correctly. In addition, now the developers need to wait for code review even before testing the code in the dev environment. This may become more important if the system we are developing cannot be tested in the local machines. This also has the potential for slowing down the development activity and sprint velocity which is counter-intuitive to agile. This can cause frustration among developers as well as reviewers. These can be avoided by creating a workflow where developers are able to create, build, deploy and test in a dev environment before requesting a code review. 

This workflow can be handled by creating an improvement in branching and merging strategies and by creating two build/release pipelines. For simplicity, let calls these pipelines as master and dev (e.g., dev for development). The master pipeline is triggered by the changes in the master branch while dev pipeline is triggered by changes in the dev branch. A developer can create a dev branch out of the master, make changes, builds and deploys using dev pipeline, most likely in the dev server, and makes needed tests and then creates a pull request for merger into master branch after review.  From this point on, the master pipeline kicks in and changes are reflected in all the environments such as dev, demo, uat or production as needed. 

With the above workflow, the code review policy was still enforced in the master branch and developers can make continuous changes in the code and tests before submitting the changes for review and ultimately to merge the changes to the master branch. 

In the minimum, we will be looking to three types of branches, namely master, dev and feature which are briefly described below:

Master Branch
o    Primary branch
o    Code review is always needed. 
o    Must be used to build/release in Prod environment
o    Consider this branch permanent.
o    There exist build and release definitions and CI enabled for all environments

Dev Branch
o    Secondary Branch
o    Ideally, be created from Master
o    Can be forced to use master
o    Does not require code review
o    Must not be used for build/release in Prod Environment
o    Consider this branch temporary. Things can be wiped out from this branch.
o    There exist build and release definition looking into this branch and therefore CI is enabled for this branch to deploy to Dev. For other environments, no automatic release from this branch.
o    This can also be used to quick hot fixes.
o    Just to note, if the merging of dev to master results in the elimination of branch (depending on code review request), creating a dev branch from master and pushing that branch to remote is sufficient.

Other Branches (as needed)
o    Feature branches, branches needed for pbi (product backlog item or an issue), etc
o    Can be created by anybody and can be merged to master or dev branch
o    Ideally, be created off master (but can be created from Dev)
o    Once coding is done, can be pushed to Dev branch for build/release in the dev environment
o    Once all coding is done, a pull request must be made for merger into the master branch
o    These are temporary in the remote. But you can keep it locally. So nature is semi-permanent.
o    With pull request, when this branch is merged to master, it is more than likely the branch will be removed from the remote.

Scenario/use case/process:
Some scenario related to development work is described below:

Working on a PBI 
  • Create a branch from master. Name it appropriately, use pbi or feature or certain keyword so that it can be identified easily.
  • Make necessary code changes.
  • Most likely there is dev branch. If a dev branch does not exist for the rep, create a dev branch. If there are no build and release definition for dev branch, make those as well.
  • Merge your branch to Dev (this will trigger build/release in Dev).
  • If all is well, create a pull request off your branch.


Working PBI and PBI has been postponed for future
  • Created a branch and merged code to Dev. The PBI was postponed for the time being. I need another Dev branch.
  • Keep the feature branch in place. Dev is going to be reset with the master branch. Changes made in dev will be gone.
  • Can do 'Cherry-Picking' to recover some changes and bring it to current code 

Working on the dev branch and need to remove all new changes  
  • Select the dev branch. Point to master branch and reset the dev branch to the selected point in the master branch.
  • If needed keep the local changes to a separate branch



Work on the dev branch, but there is no build or release pipeline for the dev branch
  • Clone the build definition. Rename the definition to *-Dev. In the repo, select 'dev' branch. Do the same in the release. Ensure automatic trigger is only limited to 'dev' environment.



The workflow above works, but with certain assumptions. It is safe to merge master to dev (if developers code is wiped off, they can merge the code again from the feature branch to dev at any time). Thus all the developers should know this possibility so that there is no confusion. However, to make the process frictionless, it would be a good idea to talk in the daily scrum. In most cases, the dev branch does not need to be reset as frequently, but it is ideal to have a current code in dev branch as well to avoid working on an older version of the code and to avoid merge conflict in the future.

If the master cannot be merged with Dev, force merge (this will wipe out conflicts and makes the Dev a copy of master). By forcing the merge, you are avoiding manually correcting the errors that are possibly not needed. The changes in dev can be re-applied from the feature branch if needed.

Git Extension (a tool which provides a much better experience than Git tools in Visual Studio) can be used to create, merge, reset and cherry-pick while implementing the branching strategy. This is free and available at https://git-extensions-documentation.readthedocs.io/en/latest/git_extensions.html