in

System.Data.SQLite

An open source ADO.NET provider for the SQLite database engine

Writing parameterized queries

Last post 09-14-2010 10:53 AM by Roan. 22 replies.
Page 1 of 2 (23 items) 1 2 Next >
Sort Posts: Previous Next
  • 11-21-2005 1:30 PM

    Writing parameterized queries

    Locked Reply Contact

    Parameterized queries are something often overlooked by developers.  It either looks confusing, or can't possibly be that important, or the developer wants more fine-tuned control over their queries, the excuses go on.  If you're even remotely interested in maximizing performance, reducing SQL injection risks, or are deciding whether or not its worthwhile to rewrite your code to use parameterized queries, then please read on!  If you already know all about parameterized queries, just skip to the end and read the syntax.  Perhaps the single greatest feature of parameterized queries is that every major database engine takes advantage of them!  What follows applies to all major database engines, not just SQLite:

    First things first.  SQL (Structured Query Language) is an interpreted language.  Every time you execute a new SQL query the contents of the query must be parsed, an execution plan developed, memory allocated, etc.  Parsing SQL is an expensive operation, though usually not as expensive as accessing the underlying data the query references.  The decision on when to use paramterized queries is often influenced by a couple of factors:

    • Does the query involve input from the user?
    • Am I executing essentially the same statement with slightly different input values repeatedly?

    If your code does either of these, you should consider using parameterized queries.  Take for example, the following loop:

    string lookupValue;
    using (SQLiteCommand cmd = cnn.CreateCommand())
    {
    for (int i = 0; i < 100; i++)
    {
    lookupValue = getSomeLookupValue(i);
    cmd.CommandText = @"UPDATE [Foo] SET [Value] = [Value] + 1
    WHERE [Customer] LIKE '" + lookupValue + "'";
    cmd.ExecuteNonQuery();

    }
    }

    Now this code may look innocent enough, but it suffers some performance penalties and some security risks.  First, the CommandText has to be re-evaluated every time the command is executed.  SQLite must parse the statement and construct a query plan 100 times in this loop.  There are also a lot of memory allocations being done here.  The previously-prepared CommandText is freed, the new CommandText allocated.  A statement is compiled and strings are concatenated causing even more allocations and deallocations.  There are also a great number of interop calls being performed behind the scenes.

    If lookupValue is unknown, provided by the user, or can be altered externally, this statement becomes a risk for an injection attack.  Imagine what would happen if the lookupValue contained the string '; DELETE FROM Foo; SELECT '

    Now put that statement together and you have:

    UPDATE [Foo] SET [Value] = [Value] + 1
    WHERE [Customer] LIKE ''; DELETE FROM Foo; SELECT ''

    Now this is a disastrous injection attack.  The very least you'll have to do to defend against this is double any single quotes that may appear in lookupValue, which of course is yet another step in the whole loop that will slow things down.

    Parameterized queries provide a means to minimize all this impact.  With a parameterized query, the CommandText is set only once at the beginning of the loop.  The lookupValue becomes a parameter, assigned within the loop, and the command is executed over and over without having to do any extra parsing.  Furthermore, since the parameter is a string variable and the SQL statement has already been prepared, it is not vulnerable to an injection attack.

    SQLite supports named and unnamed parameters.  Named parameters must appear in the SQL statement with either a $ (dollar), : (colon) or @ (at sign) prefix.  Unnamed parameters consist of a single question mark ? character.  Rewriting the above code to use a named parameter looks like this:

    using (SQLiteCommand cmd = cnn.CreateCommand())
    {
    cmd.CommandText = @"UPDATE [Foo] SET [Value] = [Value] + 1
    WHERE [Customer] LIKE @lookupValue";
    SQLiteParameter lookupValue = new SQLiteParameter("@lookupValue");
    cmd.Parameters.Add(lookupValue);
     
      for (int i = 0; i < 100; i++)
    {
    lookupValue.Value = getSomeLookupValue(i);
    cmd.ExecuteNonQuery();

    }
    }

    The same code now using an unnamed parameter (compatible with Jet/Access)

    using (SQLiteCommand cmd = cnn.CreateCommand())
    {
    cmd.CommandText = @"UPDATE [Foo] SET [Value] = [Value] + 1
    WHERE [Customer] LIKE ?";
    SQLiteParameter lookupValue = new SQLiteParameter();
    cmd.Parameters.Add(lookupValue);
     
      for (int i = 0; i < 100; i++)
    {
    lookupValue.Value = getSomeLookupValue(i);
    cmd.ExecuteNonQuery();

    }
    }

    For simplicity I didn't wrap these functions inside a transaction which would have made them significantly faster.  Even still, the amount of processing that must be done for the parameterized queries are a fraction of the original.  When performing a bulk update or insert, or executing a query using outside information, parameterized queries are a clear winner.

    Robert

     

  • 12-13-2005 2:37 PM In reply to

    Re: Writing parameterized queries

    Just a minor nit...   :)

    @ = "at", "at sign" or even "asperand".

    & = "ampersand"

    http://en.wikipedia.org/wiki/%40

    I kept seeing references to "ampersand" in conjunction with parameters and all I kept seeing were "@'s".   :)

  • 12-13-2005 2:51 PM In reply to

    Re: Writing parameterized queries

    Bah, that one escaped my grammar policing.
  • 01-28-2006 6:41 PM In reply to

    Re: Writing parameterized queries

    I'm new to ADO .Net style programming so I'm unsure if my situation would benefit from this. I was hoping you might have some insight.

    I am running very similar commands of the course of the program, however they are not in a loop condition. For example, when the user makes changes to a contact's information we run a very similar "update" command, no matter which contact they were editing.  Same goes for other tables, and with inserts.

    I'm thinking of making the SQLiteParameters private members of my class, however I would end up with a parameter for just about every column in my database :-(  and this would add an additionaly 20-30 static variables to my db class. I'm having trouble believing it would benefit me to do this.  Here is a section of code that I have:

            public void UpdateContact(Contact bob)
            {
                // Setup parameters for query.
                m_paramFname.Value = bob.FirstName;
                m_paramMname.Value = bob.MiddleName;
                m_paramLname.Value = bob.LastName;
                m_paramGuid.Value = bob.Id;

                using( SQLiteCommand cmd = m_conn.CreateCommand() )
                {
                    cmd.Parameters.Add(m_paramFname);
                    cmd.Parameters.Add(m_paramMname);
                    cmd.Parameters.Add(m_paramLname);
                    cmd.Parameters.Add(m_paramGuid);

                    cmd.CommandText = "UPDATE person SET firstName = @firstName, middleName "
                        + "= @middleName, lastName = @lastName WHERE guid = @id";

                    int numEffected = cmd.ExecuteNonQuery();
                }
            }

  • 01-28-2006 8:43 PM In reply to

    Re: Writing parameterized queries

    You don't have to (nor should you) do it that way ... Instead, build the command one time and keep the SQLiteCommand as a member of the class.  (Don't forget to dispose it when your class is no longer needed).  Your code changes to:

    private SQLiteCommand m_cmd = CreateUpdateCommand();

    public void UpdateContact(Contact bob)
    {
      m_cmd.Parameters["@firstName"].Value = bob.FirstName;
      m_cmd.Parameters["@middleName"].Value = bob.middleName;
      m_cmd.Parameters["@lastName"].Value = bob.LastName;
      m_cmd.Parameters["@id"].Value = bob.Id;

      m_cmd.ExecuteNonQuery();
    }

    private SQLiteCommand CreateUpdateCommand()
    {
      SQLiteCommand cmd = new SQLiteCommand("UPDATE person SET firstName = @firstName, middleName = @middleName, lastName = @lastName WHERE guid = @id", m_conn);

      cmd.Parameters.Add(new SQLiteParameter("@firstName"));
      cmd.Parameters.Add(new SQLiteParameter("@middleName"));
      cmd.Parameters.Add(new SQLiteParameter("@lastName"));
      cmd.Parameters.Add(new SQLiteParameter("@id"));

      return cmd;
    }

     

  • 01-29-2006 3:58 PM In reply to

    Re: Writing parameterized queries

    Ok, that makes sense. Thank you.
  • 06-15-2006 4:07 AM In reply to

    • AlexDP
    • Top 150 Contributor
    • Joined on 06-15-2006
    • Ukraine, Dniepropetrovsk
    • Posts 9

    Re: Writing parameterized queries

    But it seems you have to store not only command, but connection to, because command is executed on connection.

    And I don't know how to bind existing command to new connection.

     

  • 06-15-2006 8:53 AM In reply to

    Re: Writing parameterized queries

    To assign a command to a new connection, simply set the Connection property of the command:

    myCommand.Connection = myNewConnection;

    You can do this anytime, as long as the command isn't currently executing and doesn't currently have a datareader open on it.

    Robert

     

  • 07-14-2006 7:23 AM In reply to

    • AlexDP
    • Top 150 Contributor
    • Joined on 06-15-2006
    • Ukraine, Dniepropetrovsk
    • Posts 9

    Re: Writing parameterized queries

     rsimpson wrote:

    You don't have to (nor should you) do it that way ... Instead, build the command one time and keep the SQLiteCommand as a member of the class.  (Don't forget to dispose it when your class is no longer needed).  Your code changes to:

    I think it's much better idea to store commands like static fields and create them in static constructor of the class.

     

  • 10-12-2006 2:38 PM In reply to

    Re: Writing parameterized queries

    I don't know if I doing something wrong, but I switch my code to use parameterized queries and I see no benefit as far as processing time. There are 185 recods being inserted here and it takes nearly a minutes. Something must be wrong. Any clues?

            Dim FamCmd, MemCmd As SQLiteCommand
            FamCmd = Cnn.CreateCommand
            MemCmd = Cnn.CreateCommand

            FamCmd.CommandText = String.Format("insert into families (id, ward_id, family, phone, addr1, addr2, addr3, addr4) values (@family_id, {0}, @family, @phone, @addr1, @addr2, @addr3, @addr4);", WardUnit)
            Dim family_id As New SQLiteParameter("@family_id")
            FamCmd.Parameters.Add(family_id)
            Dim family As New SQLiteParameter("@family")
            FamCmd.Parameters.Add(family)
            Dim phone As New SQLiteParameter("@phone")
            FamCmd.Parameters.Add(phone)
            Dim addr1 As New SQLiteParameter("@addr1")
            FamCmd.Parameters.Add(addr1)
            Dim addr2 As New SQLiteParameter("@addr2")
            FamCmd.Parameters.Add(addr2)
            Dim addr3 As New SQLiteParameter("@addr3")
            FamCmd.Parameters.Add(addr3)
            Dim addr4 As New SQLiteParameter("@addr4")
            FamCmd.Parameters.Add(addr4)

            MemCmd.CommandText = "insert into members (family_id, position, name) values (@family_id, @position, @name)"
            Dim mem_family_id As New SQLiteParameter("@family_id")
            MemCmd.Parameters.Add(mem_family_id)
            Dim position As New SQLiteParameter("@position")
            MemCmd.Parameters.Add(position)
            Dim name As New SQLiteParameter("@name")
            MemCmd.Parameters.Add(name)

            swrCsv.ReadLine()     ' trash the header line
            Do While swrCsv.Peek() >= 0
                Fields = ParseCsvLine(swrCsv.ReadLine)
                'Sql = String.Format("insert into families (ward_id, family, phone, addr1, addr2, addr3, addr4) values ({0}, {1}, {2}, {3}, {4}, {5}, {6});", WardUnit, Fields(0), Fields(1), Fields(2), Fields(3), Fields(4), Fields(5))
                FamilyCounter += 1
                family_id.Value = FamilyCounter
                family.Value = Fields(0).Trim("""")
                phone.Value = Fields(1).Trim("""")
                addr1.Value = Fields(2).Trim("""")
                addr2.Value = Fields(3).Trim("""")
                addr3.Value = Fields(4).Trim("""")
                addr4.Value = Fields(5).Trim("""")
                Try
                    FamCmd.ExecuteNonQuery()
                Catch ex As Exception
                    MsgBox("Could not insert family into database: " & ex.Message, MsgBoxStyle.Exclamation)
                End Try

                Me.StatusBar.Text = Fields(0) & " added."
                mem_family_id.Value = FamilyCounter

                For i = 6 To Fields.Length - 1
                    'Sql = String.Format("insert into members (family_id, position, name) values ({0}, {1}, {2})", 0, i - 5, Fields(i))
                    position.Value = i - 5
                    name.Value = Fields(i).Trim("""")
                    Try
                        MemCmd.ExecuteNonQuery()
                    Catch ex As Exception
                        MsgBox("Could not add family member into database: " & ex.Message, MsgBoxStyle.Exclamation)
                    End Try
                Next
            Loop
  • 10-12-2006 3:01 PM In reply to

    Re: Writing parameterized queries

    You, like so many before you (and many to come after!) have forgotten the cardinal rule of SQLite ... TRANSACTIONS!  Changes are in red:

          Dim DbTransaction trans
          Try
            trans = Cnn.BeginTransaction()   
            swrCsv.ReadLine()     ' trash the header line
            Do While swrCsv.Peek() >= 0
                Fields = ParseCsvLine(swrCsv.ReadLine)
                'Sql = String.Format("insert into families (ward_id, family, phone, addr1, addr2, addr3, addr4) values ({0}, {1}, {2}, {3}, {4}, {5}, {6});", WardUnit, Fields(0), Fields(1), Fields(2), Fields(3), Fields(4), Fields(5))
                FamilyCounter += 1
                family_id.Value = FamilyCounter
                family.Value = Fields(0).Trim("""")
                phone.Value = Fields(1).Trim("""")
                addr1.Value = Fields(2).Trim("""")
                addr2.Value = Fields(3).Trim("""")
                addr3.Value = Fields(4).Trim("""")
                addr4.Value = Fields(5).Trim("""")
                Try
                    FamCmd.ExecuteNonQuery()
                Catch ex As Exception
                    MsgBox("Could not insert family into database: " & ex.Message, MsgBoxStyle.Exclamation)
                End Try

                Me.StatusBar.Text = Fields(0) & " added."
                mem_family_id.Value = FamilyCounter

                For i = 6 To Fields.Length - 1
                    'Sql = String.Format("insert into members (family_id, position, name) values ({0}, {1}, {2})", 0, i - 5, Fields(i))
                    position.Value = i - 5
                    name.Value = Fields(i).Trim("""")
                    Try
                        MemCmd.ExecuteNonQuery()
                    Catch ex As Exception
                        MsgBox("Could not add family member into database: " & ex.Message, MsgBoxStyle.Exclamation)
                    End Try
                Next
            Loop
            trans.Commit()
          Finally
            If trans != null Then trans.Dispose() '  My VB is rusty is this right?
          End Try


           

  • 10-16-2006 1:19 AM In reply to

    Re: Writing parameterized queries

    Hi, I often write code in VB ;-)

    Using trans As DbTransaction = Cnn.BeginTransaction()
        ....
        trans.Commit()
    End Using

    or

    Dim trans As DbTransaction
    Try
        trans = Cnn.BeginTransaction()
        ....
        trans.Commit()
    Finally
        If trans IsNot Nothing Then trans.Dispose()
    End Try
    Regards

    Jesús López

  • 11-02-2006 6:12 AM In reply to

    Re: Writing parameterized queries

    Awesome! Thank you both. That was SO much faster!

    So these DbTransactions, are they unique to sqlite or are they used in all sql dbs'?

    Thanks again for helping the noob
  • 11-02-2006 7:31 AM In reply to

    Re: Writing parameterized queries

    All database engines supports transactions (at least they should).

    SQLite is somehow special regarding transactions. SQLite executes much faster one single large transaction than many small transactions. This is a SQLite characteristic, other database engines might behave different.


    Regards

    Jesús López

  • 02-10-2010 6:08 AM In reply to

    Re: Writing parameterized queries

     My paramthesized queries simply do not input any values into the places in the query where the paramter is, ill post my method.

     

     Public Shared Sub SQLite_ExecuteNonQuerys(ByVal table As DataTable)
            If SQLite_Connect() = False Then
                Return
            End If

            Dim trans As DbTransaction = SQLite_Connection.BeginTransaction()
            Dim adapter As New SQLiteDataAdapter()
            adapter.SelectCommand = New SQLiteCommand(String.Format("SELECT * FROM {0}", table.TableName), SQLite_Connection, trans)

            Dim update As String = String.Format("UPDATE {0} SET ", table.TableName)
            Dim insert_columns As String = ""
            Dim insert_values As String = ""
            Dim paramaters As New List(Of SQLiteParameter)
            Dim key_index As Integer = -1
            For i As Integer = 0 To (table.Columns.Count - 1) Step 1
                If i <> (table.Columns.Count - 1) Then
                    update = update & String.Format("{0} = {1},", table.Columns(i).ColumnName, String.Format("@{0}", table.Columns(i).ColumnName))
                    insert_columns = insert_columns & String.Format("{0},", table.Columns(i).ColumnName)
                    insert_values = insert_values & String.Format("{0},", String.Format("@{0}", table.Columns(i).ColumnName))
                Else
                    update = update & String.Format("{0} = {1}", table.Columns(i).ColumnName, String.Format("@{0}", table.Columns(i).ColumnName))
                    insert_columns = insert_columns & String.Format("{0}", table.Columns(i).ColumnName)
                    insert_values = insert_values & String.Format("{0}", String.Format("@{0}", table.Columns(i).ColumnName))
                End If
                paramaters.Add(New SQLiteParameter(String.Format("@{0}", table.Columns(i).ColumnName), table.Columns(i).ColumnName))

                If table.Columns(i).Unique Then
                    key_index = i
                End If
            Next

            Dim insert As String = String.Format("INSERT INTO {0} ({1}) VALUES ({2})", table.TableName, insert_columns, insert_values)
            Dim delete As String = String.Format("DELETE FROM {0} ", table.TableName)
            If key_index <> -1 Then
                update = update & String.Format("WHERE {0} = {1}", table.Columns(key_index).ColumnName, String.Format("@{0}", table.Columns(key_index).ColumnName))
                delete = delete & String.Format("WHERE {0} = {1}", table.Columns(key_index).ColumnName, String.Format("@{0}", table.Columns(key_index).ColumnName))
            End If

            adapter.UpdateCommand = New SQLiteCommand(update, SQLite_Connection, trans)
            adapter.InsertCommand = New SQLiteCommand(insert, SQLite_Connection, trans)
            adapter.DeleteCommand = New SQLiteCommand(delete, SQLite_Connection, trans)
            For i As Integer = 0 To (paramaters.Count - 1) Step 1
                adapter.UpdateCommand.Parameters.Add(paramaters(i))
                adapter.InsertCommand.Parameters.Add(paramaters(i))
                adapter.DeleteCommand.Parameters.Add(paramaters(i))
            Next
            adapter.Update(table)
            trans.Commit()

            SQLite_Close()
        End Sub

Page 1 of 2 (23 items) 1 2 Next >
Powered by Community Server (Commercial Edition), by Telligent Systems