in

System.Data.SQLite

An open source ADO.NET provider for the SQLite database engine

Schema generating trouble?

Last post 01-23-2009 9:13 AM by Sky. 7 replies.
Page 1 of 1 (8 items)
Sort Posts: Previous Next
  • 01-23-2009 6:26 AM

    • Sky
    • Top 500 Contributor
    • Joined on 01-23-2009
    • Posts 7

    Schema generating trouble?

    Hi!

    It seems, that some trouble exists in schema generating routine (System.Data.SQLite version is 1.0.60.0).

    I wrote an example, that illustrate my problem with reading data from data reader to table, when we to deal with using left outer joins in query.

    First, lets create a test database and fiil it with test data:

    =================
    begin transaction;
    
    create table ArticleLine
    (
     al_id             integer      not null primary key autoincrement,
     al_name           varchar(100) not null
    );
    
    create table Catalogue
    (
     art_id            char(6)      not null,
     al_id             integer          null
       constraint fk_catalogue2 references ArticleLine(al_id),
     primary key (art_id)
    );
    
    insert into ArticleLine(al_id, al_name) values(1, 'Line 1');
    insert into Catalogue(art_id, al_id) values('000001', 1); 
    insert into Catalogue(art_id, al_id) values('000002', null);
    commit;
    =================
    
    Second, lets write a small example:
    using System;
    using System.Data;
    using System.Data.SQLite;
    
    namespace SQLiteTest
    {
     class Program
     {
      static void Main()
      {
       using (IDbConnection conn = new SQLiteConnection("Data Source=test.db3"))
       {
        conn.Open();
    
        string[ queries = 
        {
         "select t1.art_id, t2.al_id from Catalogue t1 " + 
           "left outer join ArticleLine t2 on t2.al_id = t1.al_id",
         "select t1.art_id, t1.al_id from Catalogue t1 " + 
           "left outer join ArticleLine t2 on t2.al_id = t1.al_id",
         "select t1.art_id, t1.al_id, t2.al_name from Catalogue t1 " + 
           "left outer join ArticleLine t2 on t2.al_id = t1.al_id"
        };
    
        foreach (string query in queries)
        {
            try
            {
                Console.WriteLine("Executing query:");
                Console.WriteLine(query);
    
                using (IDbCommand cmd = conn.CreateCommand())
                {
                    cmd.CommandText = query;
                    using (IDataReader reader = cmd.ExecuteReader())
                    {
    
                        DataTable dt = new DataTable();
                        dt.Load(reader);
                    }
                }
                Console.WriteLine("OK!");
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            Console.WriteLine("============================");
        }
       }
      }
     }
    }
    ===========================
    

    In this example we can see that "AllowDBNull" property for field " t2.al_id" detects incorrectly (first query)

    Ok, I suppose, that we can wrap-arround this by using "t1.al_id" instead of " t2.al_id"... And it really works! (second query)

    ... but, I need to obtain a "t2.al_name" from query (third query), but in datatable schema this column also has AllowDBNull=false , which is incorrect, when we use "left outer join".

    Any ideas about this?

    Of course, I can define "ArticleLine.al_name" as nullable, but it does not correspond business rules of task.

    Thanks!

    P.S. Sorry for my ugly english.. :(

  • 01-23-2009 7:43 AM In reply to

    Re: Schema generating trouble?

    First, the issue of "integer not null primary key":
    This I deliberately leave as nullable in the schema because in SQLite, you are allowed to specify NULL in such a case to have SQLite generate the rowid for you.

    Second, in the case of the left outer join:
    How do other databases handle this?  I have no knowledge of your query.  I only know what rows you've requested and from what tables they came from.  Therefore it is impossible for me to somehow figure out the *context* of your query and somehow come to the conclusion that a particular column from a particular query, when queried a particular way will be nullable or not.

     

  • 01-23-2009 8:12 AM In reply to

    • Sky
    • Top 500 Contributor
    • Joined on 01-23-2009
    • Posts 7

    Re: Schema generating trouble?

    Thanks for quick reply, Robert!

    About "integer not null primary key" - in this case its not a problem at all. Now I try only to select from tables.

    > How do other databases handle this?

    I try to check this now...

    > I have no knowledge of your query

    Why, Robert?! In my test example (in my message text) I wrote 3 queries, which I try to execute.

    No special "context" exists - you can compile the example source above and you see mentioned problem.

  • 01-23-2009 8:28 AM In reply to

    • Sky
    • Top 500 Contributor
    • Joined on 01-23-2009
    • Posts 7

    Re: Schema generating trouble?

    I'd check my example on Sybase SQL Anywhere via OdbcConnection

    In all queries art_id AllowDBNull == false, al_id AllowDBNull == true

    And al_name in third query AllowDBNull == true

    ... And such behaviour seems quite reasonable. After all in the table "Catalogue" it is specified that al_id can be null, therefore, after executing of query we can receive al_id and al_name = null in some records

    P.S. When I speek about AllowDbNull, I mean dt.Columns["al_id"].AllowDBNull property in data table in which data from reader must be written.

  • 01-23-2009 8:34 AM In reply to

    Re: Schema generating trouble?

    The reason is simple -- I can't parse your SQL.  I send the CommandText straight to the SQLite engine, which compiles it into pseudo-code.  The kind of code needed to parse your query and make intelligent decisions about what you are trying to accomplish and how to return schema would put me 50% of the way toward writing my own database engine and more than triple the codebase.

    Furthermore, consider this ... what if you were querying schema on a VIEW?  Lets say your view has LEFT JOIN's in it, but your query that you send to SQLITE is "select * from myview".  Now, I have no query text to parse that gives me anything meaningful about that view -- unless I'm smart enough to pull the view schema and parse that as well at runtime. 

    Lets get really complicated, and do an outer join against two views, each with inner joins.  Lets even say that they use different join syntax -- one uses SELECT * FROM A, B WHERE A.ID = B.ID and the other one uses INNER JOIN ...

    But wait, there's more!  Lets say one of the queries is SELECT * FROM A, B WHERE MyUserDefinedFunction(A.ID, B.ID) = 1   ... How do I deal with that without knowing what MyUserDefinedFunction is or how it works?

    That's just the very tip of the iceburg.  I didn't even get into variations involving UNION and INTERSECT.  As you can see, without implementing a full-blown SQL parser that understands context and can actually make sense of your queries (however complex they may be) I will never be able to provide you with context-based schema information.

     

  • 01-23-2009 8:41 AM In reply to

    Re: Schema generating trouble?

    Unfortunately, SQLite doesn't have the same level of internal schema support that MySQL and Sql Server have.  The reason you can get that level of schema data from MySQL is because its built into the engine.  SQLite has no built-in support for what you're trying to accomplish and it has to be "faked" by the ADO.NET provider.

     

  • 01-23-2009 9:06 AM In reply to

    • Sky
    • Top 500 Contributor
    • Joined on 01-23-2009
    • Posts 7

    Re: Schema generating trouble?

    Yes, certainly. I understand that System.Data.SQLite provider send command into sqlite-engine.

    But then a question: when DataTable. Load (reader) receives the table schema, who supplies with its information on columns?

    I believe that System.Data.SQLite, but most likely from sqlite-engine...

    Which, certainly, does complete analysis of inquiry and has the all necessary information.

    But then this problem - in sqlite-engine itself.

    The second variant is that System.Data.SQLite looks at columns which are mentioned in inquiry and their gets аttributes from related tables...

    What of variants the correct? That is the schema returns sqlite-engine, and the provider simply transfers it to the application. Or the provider himself does the query analysis?

  • 01-23-2009 9:13 AM In reply to

    • Sky
    • Top 500 Contributor
    • Joined on 01-23-2009
    • Posts 7

    Re: Schema generating trouble?

    Thanks, Robert!

    It is possible to consider that the question is solved.

Page 1 of 1 (8 items)
Powered by Community Server (Commercial Edition), by Telligent Systems