Everything I’ve read about blog posts says I need a large image to start. So, here goes…
First, I must say I’m quite humbled that anyone cared to read my first blog. I approached that last entry with a perspective that a couple dozen people would read it over the course of a few weeks, if that. As of writing this, it’s sitting at over 2,000 reads, several Twitter shares including a Microsoft MVP, an engineer on the Entity Framework team, and a co-worker asking if that was actually me out of the blue. Takeaways: check the ‘tude, and refine the message.
So, to the point. My main impression from the last article is that there’s a strong thirst for good Entity Framework resources. The issues are well-known, the frustrations are felt by many, and people want to know what to do. That’s what this entry is about.
Step 0) Generation
Brass tacks time; here are the packages I’ll be working with.
(TLDR; GitHub repo if you want to play with it yourself.)
Microsoft.EntityFrameworkCore.SqlServer The main package that brings in the EF Core runtime as well as the components needed to work with SQL Server.
Microsoft.EntityFrameworkCore.InMemory Provides InMemory support allowing us to do easy as pie mocking. It should be noted, though, that it doesn’t enforce relational constraints. Since its most popular scenario so far is with testing, that’s not an issue since the whole point of tests is to verify things. Even so, the capability for a fast, albeit volatile, memory database is very much there. Imagination is the only real limit.
Microsoft.EntityFrameworkCore.SqlServer.Design Microsoft.EntityFrameworkCore.Tools These are necessary to reverse engineer an existing database which is my preferred approach. I much prefer to work with SQL Server in all it’s ugly glory. Being able to define what you want your database to be (re: Let the Database be a Database) and letting the runtime suck in all the detail it can handle is a quick way to ensure you’re getting the most out of your code-first implementation. New version of EF comes out? Import it again. But, this is like buying a suit at a store. It’ll definitely work. It’ll even look good, but it will need some tailoring, and cufflinks are separate. In an actual product, remember that partials are your friend.
System.Reactive Providing the most unique feature is the .Net Reactive Extensions. These can be used in tons of different ways, but will work for demonstrating ETL capabilities.
Z.EntityFramework.Plus.EFCore EF is still lacking in declarative actions that issue commands at the data store. This isn’t bad as being agnostic requires certain trade offs. Yet, EF is extensible which is what lets products like this exist. This product lets us have Update and Delete capabilities without needing to interact with entities at all. The Z libraries are solid, but I do have a problem with the pricing of their commercial bulk insert offering. What I’ll show here will get you most of what their bulk insert offering does albeit probably not as polished. This is a free blog, after all.
You can install these with the following line:
“Microsoft.EntityFrameworkCore.SqlServer”, “Microsoft.EntityFrameworkCore.InMemory”, “Microsoft.EntityFrameworkCore.SqlServer.Design”, “Microsoft.EntityFrameworkCore.Tools”, “System.Reactive”, “Z.EntityFramework.Plus.EFCore” | foreach {Install-Package $_}
Unfortunately, the take with .Net Core packages means all that will bloat to like 70 packages total with all its dependencies. To match the database I’ll be using, run the script EfStore.create.sql. The point of this entry isn’t to go over the schema aspects but how Entity Framework can live in an application and how we can interact with it without it being a kludgey nightmare. Alas, I loathe two table databases “examples” with a passion.
The command I used to import the EfStore database is the following:
Scaffold-DbContext “Server=(local);Database=EfStore;Trusted_Connection=True;” Microsoft.EntityFrameworkCore.SqlServer -OutputDir StoreData
Since partials are our friends, lets partial out a shard of the context as {context_name}.Customizations.cs. Because we do this, we should watch out for the pregenerated OnConfiguring method in the context. You can delete this.
hoagsie/BetterEntityFramework_BetterEntityFramework - Some examples on how to more cleanly work with Entity Framework_github.com
Step 1) Isolation
I completely agree with the notion that we shouldn’t need to litter our code with DbContext references. Why would that ever be necessary? Yet, I still stand by my statement that the repository pattern is a throne of lies. It goes too far and lobotomizes Entity Framework’s potential.
The primary point of contact can take many forms. In my example, since I only work with a SQL Server store, I called it DataService. In an application dealing with many providers, it could likely become a factory or some sort of look up that would know how to expose the DbContext to the caller.
hoagsie/BetterEntityFramework_BetterEntityFramework - Some examples on how to more cleanly work with Entity Framework_github.com
As you can wrap the DbContext, you can also wrap the DbContextOptionsBuilder. I call this the DataOptionsBuilder. Yes, I do some implicit operator overloading to let the naked DbContextOptionsBuilder fit into the DataService’s methods without needing to do explicit casting and having it look weird, but it works well for conceptual demo purposes. In a production application you should put this away in a factory that the service calls or can be passed in if you want to invert your dependency.
hoagsie/BetterEntityFramework_BetterEntityFramework - Some examples on how to more cleanly work with Entity Framework_github.com
After these, you have all the layering you need between your application and Entity Framework. You can completely rip it out and as long as the contracts remain the same, you don’t need to make any changes to your application code. This also shows why I had the ire I did in my prior blog entry.
You don’t need these cookie cutter pieces dotting your applications like minefields for every little facet you want to expose. Save yourself some work and use the framework to your advantage. Granted, some people don’t like tying themselves too much to a particular framework. Protip: you lost that battle like five times over by time you write “hello, world” in a .net console app. If you instead want to be able copy/paste (with little modification) from one framework to another, there’s still no reason you can’t do that. Such a thing takes an incredible amount of work no matter what even if you only ever code in primitives and arrays. Things like this also make me angry at Visual Studio WCF proxy generation. Rolling a manual proxy factory is easy, gets you greater fidelity, and is much more consistent not to mention much lighter than the half dozen files Visual Studio plops into your solution. I digress.
Step 2) Extension
You might think the above is great alone. You have Entity Framework, and it’s not littering your code like your rude friend who never cleans up after himself when he visits. Yet, it’s wholly useless for what we need and doesn’t hold up to the claims I made in the last post. For that we need two things to meet the criteria I set out.
The first part of this, DbContextExtensions, delivers a quality of life component in ClearCache methods and the show horse that destroys “EF can’t” claims, BulkInsert. The ClearCache methods are straight-forward. One takes a selector so you can choose what to get rid of. The other clears everything.
hoagsie/BetterEntityFramework_BetterEntityFramework - Some examples on how to more cleanly work with Entity Framework_github.com
Looking at the code you might go “but hoagsie, bro, you said IEnumerable was bad.” No, I didn’t. I said to break the love affair with it. We don’t want to cart around a collection of objects that might be thousands or millions of items large. We can still delay enumeration until we actually want to do something with it which is the purpose of IEnumerable. Also, this reinforces an important implicit paradigm I’m advocating: IQueryable is for fetching data and messing with it mid-execution whereas IEnumerable is for managing data we already have and can work with it in a simple manner.
“But hoagsie, bro, don’t repos do that?” No, they don’t. As I illustrated in my last entry, repositories impose restrictions upon the data you serve back to the application and are resistant to change. Also, while today you need to select some products but do some processing on all them in another module, if a junior developer needs to refactor that module four months later to skip the first 30 for whatever reason, the IQueryable can roll with this. Repositories can’t unless you go back and change the part that calls the repository.
The BulkInsert method works with SqlBulkCopy as I espoused. That shouldn’t be a surprise. Something that might be a surprise is the dedicated SqlConnection I setup for it. In my tests, the bulk copy object kept closing the underlying connection causing issues. So, we can create another connection. It is still streamed data so you can still shovel tons of data through it and you won’t be creating several connections. You have two and that’s it.
The crème de la crème of the BulkInsert is what the Reactive Extensions give us. They provide a “just a few lines” way to create powerful, expressive transformations to objects that stream through the method. You have to be mindful of what you do in the transformation or else you’ll still put your performance in the trash. Still, I’ve seen this chew through records like nobody’s business. Put the application that uses this on the network close to the database to cut latency and watch it scream.
I also provided a quick and dirty BulkHelper to provide a way to hook into the update notifications the SqlBulkCopy object provides.
hoagsie/BetterEntityFramework_BetterEntityFramework - Some examples on how to more cleanly work with Entity Framework_github.com
Lastly, the part that ties the implementation together is the QueryableDataReader. This lets the IDataReader overload of the bulk copy object understand our IQueryables and can subsequently handle anything we throw at it.
hoagsie/BetterEntityFramework_BetterEntityFramework - Some examples on how to more cleanly work with Entity Framework_github.com
Last but not least, we have our IQueryableExtensions. These utilize the Z lib to gracefully wrap the
Where(Expression<Func<TEntity, bool>>).Update(Expression<Func<TEntity, TEntity>>)
and
Where(Expression<Func<TEntity, bool>>).Delete()
features of the package without betraying it comes from somewhere else.
hoagsie/BetterEntityFramework_BetterEntityFramework - Some examples on how to more cleanly work with Entity Framework_github.com
With these extensions, the code can still be almost completely ignorant of Entity Framework and the Z lib. Let’s look at the using block in Program.cs.
using System.Linq;using System.Reactive;using BetterEntityFramework.Extensions;using BetterEntityFramework.StoreData;using Microsoft.EntityFrameworkCore;using Data = BetterEntityFramework.DataService;using IsolationLevel = System.Data.IsolationLevel;
The only reason the “using Microsoft.EntityFrameworkCore;” line is in there is because I call out to the builder options to show how we can interact with them. Putting that in a factory or a default provider implementation would leave the application aspects of the code 100% ignorant of what’s going on. Note that this is different from not pretending Entity Framework isn’t a dependency. It is. It always has been even with repositories. But, we don’t need to think about it and can let other components manage those concerns for us. That’s what it’s all about.
I did not create a “working” demo but instead opted for a recipe like scramble of examples so you can pick and choose and adapt the components to whatever you’d want to do without needing to scrape out my scenario code.