EF.Core Multithreading The timeout period elapsed

0

I am running a BackgroundTask in my ASP.NET Core 5 app which creates some entities inside my database. My idea was to run it in multiple tasks to speed up the generator.

    public class GeneratorService : BackgroundService
    {
        private const int TaskCount = 8;

        private uint _index;
        private readonly List<Task> _tasks;
        private readonly IServiceScopeFactory _serviceScopeFactory;

        public GeneratorService(IServiceScopeFactory serviceScopeFactory)
        {
            _serviceScopeFactory = serviceScopeFactory;

            _tasks = new List<Task>();
        }

        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            while (stoppingToken.IsCancellationRequested == false)
            {
                for (uint i = 0; i < TaskCount; i++)
                {
                    var i1 = i;

                    var task = new Task(async () => await Generate(_index + i1, stoppingToken));

                    task.Start();
                    _tasks.Add(task);
                }

                Task.WaitAll(_tasks.ToArray(), stoppingToken);

                _tasks.Clear();

                _index += TaskCount;
                await Task.Delay(10, stoppingToken);
            }
        }

        private async Task Generate(uint index, CancellationToken stoppingToken)
        {
            using var scope = _serviceScopeFactory.CreateScope();

            var context = scope.ServiceProvider.GetService<DataContext>();
            var repository = scope.ServiceProvider.GetService<Repository>();
            var generator = scope.ServiceProvider.GetService<Generator>();

            // A simple return await FirstOrDefaultAsync(e => e.Number == index)
            var entity = await repository.GetByNumberAsync(index);
            if (entity== null) ...
                

            generator.GenerateChildren(entity);

            await context.SaveChangesAsync(stoppingToken);

            Console.WriteLine($"Entity {index} generated");
        }
    }

The generator loop runs for about 100 - 200 entities, but then it starts throwing me exceptions:

fail: Microsoft.EntityFrameworkCore.Query[10100]
      An exception occurred while iterating over the results of a query for context type 'Namespace.DataContext'.
      Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
       ---> System.ComponentModel.Win32Exception (258): Der Wartevorgang wurde abgebrochen.
         at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__169_0(Task`1 result)
         at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
         at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
      --- End of stack trace from previous location ---
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(DbContext _, Boolean result, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
      ClientConnectionId:f8508f09-1298-487c-8513-93bd4289014e
      Error Number:-2,State:0,Class:11
      Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
       ---> System.ComponentModel.Win32Exception (258): Der Wartevorgang wurde abgebrochen.
         at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__169_0(Task`1 result)
         at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
         at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
      --- End of stack trace from previous location ---
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(DbContext _, Boolean result, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
      ClientConnectionId:f8508f09-1298-487c-8513-93bd4289014e
      Error Number:-2,State:0,Class:11

I am registering the DataContext in an extension method:

        public static IServiceCollection ConfigureDatabase(this IServiceCollection services,
            IConfiguration configuration)
        {
            
            var connectionString = // I get my connection string via a json file

            services.AddDbContext<DataContext>(options =>
            {
                options.UseSqlServer(connectionString);
                options.EnableSensitiveDataLogging();
            });

            return services;
        }

This is a SQL Server Database running on my localhost. If I am right, the DataContext is registered as a Scoped Service. So if I am creating a scope inside the BackgroundTask, it will automatically dispose the DataContext after the Task completed.

Or is this a SQL Server related issue where it could not handle that many requests simultaneously?

c#
sql-server
multithreading
entity-framework
asp.net-core
asked on Stack Overflow Mar 17, 2021 by Daniel DirtyNative Martin • edited Mar 17, 2021 by Dale K

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0