One big table vs. small ones – multi language application, 1 500 000 rows per each language

Last post 07-06-2009 2:17 AM by MarwCzech. 5 replies.

Sort Posts:

  • One big table vs. small ones – multi language application, 1 500 000 rows per each language

    07-04-2009, 4:30 AM
    • Member
      5 point Member
    • MarwCzech
    • Member since 10-23-2006, 1:11 PM
    • Posts 3

    Hi all,
    I’m trying to do an optimal database design and I’m stuck. In a nutshell what I have and what I’m looking for: 
    - There is a multi language (4 countries) application
    - For each language I need to store 1 500 000 rows per year
    - The table has only 4 columns and one FK
    - There is a view selecting in the actual country: employee, sum of top 10 salaries, count of work shift in a month
    - Insert into DB is 1 a day, selecting from the view is 1000 a day
    The question:
    Shall I store data in one big table with 4 500 000 rows per year and select with language condition (Sample 1) or separate data into 4 small tables (Sample 2)?
    Sample 1:
    ------------------------------------------------------------------
    T_Results
    ---------------
    IdResult
    IdImport
    Employee
    Salary
    T_Import
    ----------------
    IdImport
    ImportDate
    Language
    Sample 2:
    -----------------------------------------------------------------
    T_Results_EN
    ----------------
    IdResult
    IdImport
    Employee
    Salary
    T_Import_EN
    ----------------
    IdImport
    ImportDate
    T_Results_DE
    ----------------
    IdResult
    IdImport
    Employee
    Salary
    T_Import_DE
    ----------------
    IdImport
    ImportDate
    Thanks a lot!
    Marw

    Hi all,

    I’m trying to do an optimal database design and I’m stuck. In a nutshell what I have and what I’m looking for: 

    • There is a multi language (4 countries) application.
    • For each language I need to store 1 500 000 rows per year.
    • The table has only 4 columns and one FK.
    • There are 4 views selecting from the countries: employee, sum of top 10 salaries, count of work shift in a month.
    • Insert into DB is 1 a day, selecting from the view is 1000 a day.

    The question:
    Shall I store data in one big table with 4 500 000 rows per year and make the views with language condition (Sample 1) or separate data into 4 small tables (Sample 2)?


    Sample 1:
    ----------------------------------------

    T_Results
    IdResult
    IdImport
    Employee
    Salary

    T_Import
    IdImport
    ImportDate
    Language


    Sample 2:
    ---------------------------------------- 

    T_Results_EN
    IdResult
    IdImport
    Employee
    Salary

    T_Import_EN
    IdImport
    ImportDate
    Language


    T_Results_DE
    IdResult
    IdImport
    Employee
    Salary

    T_Import_DE
    IdImport
    ImportDate
    Language


    Thanks a lot!
    Marw


  • Re: One big table vs. small ones – multi language application, 1 500 000 rows per each language

    07-04-2009, 9:28 AM
    Answer
    • Star
      8,486 point Star
    • integrasol
    • Member since 06-05-2009, 3:18 PM
    • Esbjerg, Denmark
    • Posts 1,596

    I think this is hard to call, especially without doing any load and performance testing. However, upfront I'd go with the single table solution and an associated  language table. I would then establish a baseline and continue to monitor the performance, and if need to be, consider a horizontal partitioning of the rows, potentially based when it was inserted or similar.

    Thanks

    Carsten

    Please click Mark as Answer if this post is of help to you. :-)



    My Blog
  • Re: One big table vs. small ones – multi language application, 1 500 000 rows per each language

    07-05-2009, 12:55 AM
    • All-Star
      30,735 point All-Star
    • Naom
    • Member since 12-31-2007, 2:08 PM
    • Wisconsin
    • Posts 6,809

     I agree with the previous poster - to me the problem sounds like a single table partitioned by language.

    Looking for a job opportunity.

    Beware of bugs in the above code; I have only proved it correct, not tried it.
    (Donald Knuth)

    Visit my blog

    PluralSight Learning Library
  • Re: One big table vs. small ones – multi language application, 1 500 000 rows per each language

    07-05-2009, 4:55 AM
    • Star
      12,577 point Star
    • malcolms
    • Member since 06-12-2008, 4:38 AM
    • Melbourne, Australia
    • Posts 2,062

    It would be a good option to split this table up into smaller tables.  That way there's less data to search through so your performance will increase.

    Just make sure you have your clustered and non clustered indexes as without them, you'll have a heap and CRUD work will be painful.

    Sincerely,
    Malcolm Sheridan

    Microsoft Certified Solution Developer
    Please remember to click "Mark as Answer" on the post that helps you, and to click "Unmark as
    Answer" if a marked post does not actually answer your question.
  • Re: One big table vs. small ones – multi language application, 1 500 000 rows per each language

    07-05-2009, 5:14 AM
    • Star
      8,486 point Star
    • integrasol
    • Member since 06-05-2009, 3:18 PM
    • Esbjerg, Denmark
    • Posts 1,596

    I disagree with splitting into smaller tables, but I certainly agree that a clustered index should be added, if not already, to the tables, preferably the ID columns, and non-clustered indexes for any field that is searchable, because of the number of rows.

    Thanks

    Carsten

    Please click Mark as Answer if this post is of help to you. :-)



    My Blog
  • Re: One big table vs. small ones – multi language application, 1 500 000 rows per each language

    07-06-2009, 2:17 AM
    • Member
      5 point Member
    • MarwCzech
    • Member since 10-23-2006, 1:11 PM
    • Posts 3

    Hi all,
    thanks for all the useful opinions. I have decided to store the data in a big table. If I’ll see the performance of the Views is not optimal I’ll try to use 4 tables instead the Views and update them once a day.

    Thanks for help.

    Marw

Page 1 of 1 (6 items)