Implementing Parallel Execution in C#

It is quite common to have usecases where we need to parallel execution to have over items in a list and wait for the whole process to be complete. C# allows tasks to achieve this result. But there are many ways to do the same and here I am just exploring some of the ways, I have done it in my projects.

In the below example, we are using a ConcurrentBag list to store the content returned by the async function ProcessContent. This works and fetched consistent results. But I am not sure on the thread exhaustion since we are using Task.Run inside the lambda. Task.Run is usually used to make sure a new thread is used to execute the function call based on threads availability

var fullList = new ConcurrentBag<SomethingViewModel>();

var tasks = new List<Task>();
contents.ForEach(content => tasks.Add(Task.Run(async () =>
{
    SomethingViewModel contentComplete = await ProcessSomething(content, token);
    if (contentComplete != null)
    {
        fullList .Add(contentComplete );
    }
})));

This method uses projection instead of forEach used above. Tasks are then assigned to an array and then projected again to a list based on null check

var tasks1 = contents.Select(x => ProcessContent(x, token))
                .ToList();
SomethingViewModel[] result = await Task.WhenAll(tasks1);
IEnumerable<SomethingViewModel> fullResult = result.Where(x => x is not null);

Third method involves using the Paralled.ForEachAsync method. This works, but had the least response times when testing for performance.

ParallelOptions parallelOptions = new()
            {
                MaxDegreeOfParallelism = 5
            };

            var fullContents = new ConcurrentBag<SomethingViewModel>();

            await Parallel.ForEachAsync(contents, parallelOptions, async (content, token) =>
            {
                SomethingViewModel wikiComplete = await ProcessSomething(content, workplaceContext, userToken, userGroups);
                if (contentComplete != null)
                {
                    fullContents.Add(contentComplete);
                }
            });

The above 3 methods worked. First 2 methods dont give the option to control the number of parallel threads and hence might cause memory issues. At the same time, when we have I/O operations like calling external apis, saving to database, the default safe way is to use the Task.WhenAll approach. I ended up using Option 2, but might revisit this later based on memory usage analysis from production.