Integrating Serilog with Entity Framework and custom Entities

Introduction

Logging is an essential part of any application, as it helps in monitoring, troubleshooting, and keeping track of the application’s behaviour. Serilog is a popular logging library for .NET applications, known for its simple configuration and rich set of features. When combined with Entity Framework Core, it can provide detailed insights into database operations. This article outlines a basic setup of Serilog with Entity Framework Core using custom entities.

Implementation

The aim of this article is to include a username when logging events. This will allow us to filter and search the logs or simply retain the information for future reference. To achieve this, we will create a custom entity that extends Serilog’s LogEvent.cs with some additional fields:

public class LogEntity 
{
    [Key]
    public int Id { get; set; }
    public string Message{ get; set; }
    public string? UserName{ get; set; }
}

Following the standard Serilog setup, we use the LoggerConfiguration to configure the basics. We begin by importing settings from appsettings.json into a configuration object. Additionally, we specify a context, which we will discuss later. Next, we need to set up our database connection; this configuration may vary depending on your setup. The key aspect for this article is the GetColumnOptions() method, which we will also create.

var logger = new LoggerConfiguration()
    .ReadFrom.Configuration(configuration)
    .Enrich.FromLogContext()
    .WriteTo.MSSqlServer(
        connectionString: configuration.GetConnectionString("DefaultConnection"),
        sinkOptions: new MSSqlServerSinkOptions
        {
            TableName = "Logs",
            SchemaName = "dbo",
            AutoCreateSqlDatabase = false,
            AutoCreateSqlTable = true,
        }, 
        columnOptions: GetColumnOptions() // created in next code block
    ).CreateLogger();
builder.Logging.ClearProviders();
builder.Logging.AddSerilog(logger);
builder.Services.AddSingleton<Serilog.ILogger>(logger);

With ColumnOptions, we can customize Serilog’s standard column setup and add our own columns. To do this, we simply add a list of SqlColumn objects to the AdditionalColumns property on ColumnOptions.

// can be created in Program.cs or any other file
ColumnOptions GetColumnOptions()
{
    var columnOptions = new ColumnOptions();
    columnOptions.AdditionalColumns = new List<SqlColumn>
    {
        new SqlColumn { 
            DataType   = SqlDbType.VarChar,  
            ColumnName = "UserName",     
            DataLength = 250, 
            AllowNull  = true },
        // add more columns here if needed
    };
    return columnOptions;
}

To ensure compatibility with Entity Framework and migrations, we need to prevent Entity Framework from creating the table, as Serilog will handle this. This can be easily achieved by using the ExcludeFromMigrations function on the table within the OnModelCreating method in your database context.

DatabaseContext.cs

public class DatabaseContext : DbContext
{
  public DbSet<LogEntity> Logs { get; set; }
  protected override void OnModelCreating(ModelBuilder builder)
  {
    base.OnModelCreating(builder);
    builder.Entity<LogEntity>()
      .ToTable("Logs", table => table.ExcludeFromMigrations());
  }
}

Utilisation

To populate this new column, we have a few options:

1. Create middleware that sets the username in the log context for the duration of the request.
2. Set the username within a using scope.
3. Directly insert the username into a log function call.

Using ASP.NET Core Identity or any other identity framework that provides the user context is likely the easiest approach. It could look something like this:

LogContextMiddleware.cs

public class LogContextMiddleware(
  Serilog.ILogger logger, 
  RequestDelegate next)
{
    public async Task Invoke(
      HttpContext context, 
      UserManager userManager)
    {
        var user = await userManager.GetUserAsync(context.User);
        using (LogContext.PushProperty("Method", context.Request.Method))
        using (LogContext.PushProperty("UserName", user?.Email ?? "Anonymous"))
        using (LogContext.PushProperty("UserId", user?.Id ?? "Anonymous"))
        {
            await next(context);
        }
    }
}

Program.cs

app.UseMiddleware<LogContextMiddleware>();

Using middleware like this also allows us to set additional properties for future use, such as the HTTP method from HTTPContext.Request or the UserId.

Context Overriding 

If you want to override properties while logging, you can use the ForContext method on the Logger object. For instance, if you need to set the UserName to “System” when running scheduled jobs, you can manually set this value easily.

logger.ForContext("UserName", "System")
      .ForContext("Item", "Something")
      .Information("{UserName:l} created {Item}");

Maintaining a standardized naming scheme for these variables can be tedious, so an alternative approach is to create extension methods for the ILogger to add specific context:

public static class LogExtensions
{
    public static ILogger AddItem(this ILogger logger, string itemName)
    {
        return logger.ForContext("Item", itemName);
    }
}

Using the function,

logger.AddItem("something").Information("{UserName:l} created {Item}")

Formatting the Log

We have several formatting options available, but let’s start with the Serilog Properties column. By default, this is an XML object that contains all properties, contexts, and rendering options set during logging. If we prefer this as a JSON object, we can replace the Properties with LogEvents in the GetColumnOptions method like this:

columnOptions.Store.Add(StandardColumn.LogEvent);
columnOptions.Store.Remove(StandardColumn.Properties);

When Serilog creates the Message column from the MessageTemplate provided in logger.Information(), it encloses the text in quotes. To prevent this, simply add “:l” after the property name like this:

logger.Information("{UserName} logged in");
// "John Doe" logged in
logger.Information("{UserName:l} logged in");
// John Doe logged in

Minimum Level

We can also disable Microsoft’s built-in information logging, which often includes a lot of unrelated information. In our case, we are only interested in error messages. This can be configured in the appsettings.json file along with other settings.

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Error",
      }
    }
  },
  "ConnectionStrings": {
      "DefaultConnection": "..."
  }
}

Reading the logs

Taking the help of this additional data, we can easily create an overview of our logs for administrators in the frontend. In our repository, we return logs from the DatabaseContext like this. We can also order them from newest to oldest and specify the number of logs to retrieve.

public class LogRepository(DatabaseContext context) : ILogRepository
{
  public<IEnumerable<LogEntity>> GetLogs(int amount)
  {
    return context.Logs
      .OrderByDescending(log => log.TimeStamp)
      .Take(amount)
      .ToList();
  }
}

Conclusion

This basic setup effectively captures user activities and error messages, providing clear insights into what went wrong. By storing the UserId, we can easily filter logs by user, making it simple to track individual user actions. Moreover, this setup is flexible and can be easily expanded to incorporate additional data sources, enhancing its utility and scope.