Dapper - find items in list that are not in table

1

I have a list of IDs (dynamically generated), and I want to find which of them do not exist in a table.

I'm using Dapper and Microsoft SQL Server.

Dapper allows IEnumerable parameters to be passed into queries and used as variables, which is nice, but I can't seem to wrangle it to work for the scenario of selecting things from that list.

Here is my code, showing two things I attempted (both of which do not work), and the Attempt3 which is the ugly solution I am contemplating (would perform poorly).

using Dapper;
using System;
using System.Data.SqlClient;
using System.Linq;

namespace DapperTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var connectionString = $"Initial Catalog=master;Integrated Security=True;";
            using (SqlConnection con = new SqlConnection(connectionString))
            {
                //Here's my overall problem:
                // Assume a table X exists with a lot of rows in it.
                // Now I have a list of IDs, which IDs in that list are not in table X?
                //Note: I will spoof X with a CTE in each query. Also, the IDs are uniqueidentifiers in the real table, 
                // but I'll use ints for this simple demo.

                Attempt1(con);

                Attempt2(con);

                Attempt3(con);
            }
        }

        //This fails because I cannot select from the @Ids parameter (what I would like to do, but syntax does not support it)
        //System.Data.SqlClient.SqlException (0x80131904): Incorrect syntax near ','
        static void Attempt1(SqlConnection con)
        {
            int[] idsToCheck = new[] { 1, 42, 112 };
            string sql = @"
                WITH X(Id) AS (SELECT 112 UNION SELECT 200)
                SELECT * FROM @Ids 
                WHERE Id NOT IN (SELECT Id FROM X)
            ";
            try
            {
                var result = con.Query(sql, new { Ids = idsToCheck });
                Console.WriteLine("Attempt2: " + string.Join(",", result));
            }
            catch (Exception e)
            {
                Console.WriteLine("Attempt1 - " + e.Message);
            }
        }

        //Here I tried to stuff the list into a table variable and then select from it.
        //Fails with this error:
        // An enumerable sequence of parameters (arrays, lists, etc) is not allowed in this context
        static void Attempt2(SqlConnection con)
        {
            int[] idsToCheck = new[] { 1, 42, 112 };
            string sql = @"
                DECLARE @Tmp TABLE ([Id] int);
                INSERT INTO @Tmp VALUES (@Id);

                WITH X(Id) AS (SELECT 112 UNION SELECT 200)
                SELECT Id FROM @Tmp WHERE Id NOT IN (SELECT Id FROM X)
            ";
            try
            {
                //note I pass in an IEnumerable directly as the second param
                var result = con.Query(sql, idsToCheck.Select(i => new { Id = i }));

                //but in the "Execute" context the below call works just fine - but I can only get back an affected row count from Execute (this frustrates me).
                //var result = con.Execute(sql, idsToCheck.Select(i => new { Id = i }));

                Console.WriteLine("Attempt2: " + string.Join(",", result));
            }
            catch (Exception e)
            {
                Console.WriteLine("Attempt2 - " + e.Message);
            }
        }

        //This works but is *very* undesirable because it fetches back a lot of data (everything in X).
        static void Attempt3(SqlConnection con)
        {
            int[] idsToCheck = new[] { 1, 42, 112 };
            string sql = @"
                WITH X(Id) AS (SELECT 112 UNION SELECT 200)
                SELECT Id FROM X
            ";
            try
            {
                var allIdsInX = con.Query<int>(sql, null);
                var result = idsToCheck.Except(allIdsInX);

                Console.WriteLine("Attempt3: " + string.Join(",", result));
            }
            catch (Exception e)
            {
                Console.WriteLine("Attempt3 - " + e.Message);
            }
        }
    }
}
c#
sql-server
dapper
asked on Stack Overflow Feb 17, 2021 by Anssssss

1 Answer

0

@asawyer, @Charlieface - here's what the TVP solution ends up looking like. More code that I'd like, but not terrible. If I make a few PLACEHOLDERListTableType table types for different ID types (UNIQUEIDENTIFIER, INT, whatever), then treat those as "standard", I guess I can call it reusable.

I still would like a more reusable solution that doesn't require custom types to be created in each db I use it on. So I'll not mark this the answer. I'm hoping someone knows some Dapper/SQL trickery that could solve this more simply.

/*
    Created this table valued type, now try using it.
        CREATE TYPE [dbo].[IntListTableType] AS TABLE 
        (
            [Id] INT NOT NULL,
            PRIMARY KEY ([Id])
        )
*/
static void Attempt4(SqlConnection con)
{
    int[] idsToCheck = new[] { 1, 42, 112 };
    string sql = @"
        WITH X(Id) AS (SELECT 112 UNION SELECT 200)
        SELECT Id FROM @Ids WHERE Id NOT IN (SELECT Id FROM X)
    ";
    try
    {
        var result = con.Query<int>(sql, new { Ids = GetIntListTableValuedParameter(idsToCheck) });

        Console.WriteLine("Attempt4: " + string.Join(",", result));
    }
    catch (Exception e)
    {
        Console.WriteLine("Attempt4 - " + e.Message);
    }
}

static SqlMapper.ICustomQueryParameter GetIntListTableValuedParameter(IEnumerable<int> ids)
{
    var dt = new DataTable();
    dt.Columns.Add("Id", typeof(int));
    foreach (var id in ids)
    {
        dt.Rows.Add(id);
    }
    return dt.AsTableValuedParameter(" [dbo].[IntListTableType]");
}
answered on Stack Overflow Feb 17, 2021 by Anssssss

User contributions licensed under CC BY-SA 3.0