0

Ninox file format

I was looking at a Ninox database file in TextEdit and it looks a lot like JSON. I've been around the forum long enough to see the speculation regarding the database engine that Ninox is built on and have participated in the speculation myself. In spite of the somewhat popular belief that it's base on SQLite and the bitching about the lack of SQL commands, it appears to be based on a NoSQL table product by Google...

 

https://ninox.com/en/forum/technical-help-5ab8fe445fe2b42b7dd39ee7/export-query-ninox-database-5b47009ab7638b1c06fb1933

 

Talk about an old thread!

 

An observation about making modifications to a Ninox database. There doesn't appear to be any garbage collection when modifications are made. For example, if you change the data in a field, the old data is still in the database file and the new information is just appended to the file. Same goes for something like moving columns. That information is stored in the database file and the old information is not deleted. Since they aren't doing any garbage collection, they really should add a "Pack" or "Vacuum" command to "My Databases" or include it in the Ninox language. In the mean time, you can duplicate your database and get the same result, but you'll need to delete the old one and rename the new one. You might be amazed at how much the database size decreases.

11 replies

null
    • Mel_Charles
    • 2 yrs ago
    • Reported - view

    Hi Sean - That is really interesting. I don't understand why the old data is simply left in the files. I've used several other databases from 1986 onward and they all have (had) data management built in re old data!. This probably goes some way as to why I am having issues with one of my main DB's that has the 800k records in a table - This data is constantly updated daily. If the old crap if left hidden then no wonder the DB is bloated!.

    Were you able to ascertain if deleted records are truly being deleted or just flagged?

    Totally agree - a compact aka remove crap button/option would be great! Jeez this if fundimental stuff!

    Also explains what is going on when I duplicate the DB evert few months and rename (as you suggest above ) the file size drops!! Now I know why - Thank you

    cheers Mel

    • Mel_Charles
    • 2 yrs ago
    • Reported - view

    Under the Options tab (beside the "Data model" tab) ther are "Rebuild index" and "Reindex files" buttons? What exactly does each do? Hard to tell as when they both bring up a dialog saying it will take some time, but there is never a progress bar or any other indication of when/if it is done. Why?

    From running other database systems over many years. My guess insofar that the reindex allows the system to get rid of any holes left in the system or issues with stuck records!

    Lets says you have 3 records - called rec1, rec2 and rec3 and you delete rec2. Thus - This record should be gone - but has it or is it’s just flagged internally and no longer visible to the user. Re index would get rid of the deleted record once and for all rec3 that had an index of 3 would now have an index of 2.

    I have asked Ninox previously on a couple of occasions to fully explain all 3 options inc rebuild thumbnails, but to date have never received an answer

    • Sean
    • 2 yrs ago
    • Reported - view

    Hi Mel,

    The only thing that came to mind regarding leaving the old data in the file is a long awaited feature request... Undo. I have no idea if that is their intent, but in the mean time, a "Rebuild database" option would be nice.

     

    I'm able to look at the database files on the Mac app, but it doesn't have "Rebuild index", "Rebuild thumbnails" or "Reindex files" options.

    • Sean
    • 2 yrs ago
    • Reported - view

    I just tested for deleted records and the information is still there, but a flag is added...

    rA1:null

    • Mel_Charles
    • 2 yrs ago
    • Reported - view

    Sean the rebuild buttons are in the cloud version - Only whenever I tested them (even screwing my data to test! in a play copty of course) not seemed to happen.

    now calle this co incendence but I have just receied a reply to my emails to support (Hmmm ?)

    Here is Ninox answer

     

    Dear Mel,

    These buttons will clean the Index file in the database if they have had a lot of changes with setting the indexes while building the database and if you have a lot of traffic (inserting and deleting records) in your database.

    Rebuild index and Reindex files - will rebuild the search indexes and will speed up the search of big databases. The indexing function should be used by experienced Ninox users when search functions or select are running very slow.
    These two buttons will recreate all currently necessary indexes for fields (if you have set indexes on fields) and for your attachments.

    Rebuild thumbnails - May be used when the thumbnails from the images are not showing correctly, which can be caused by the same reason as the index slow performance.

    There you go - Now we know!!

    • Sean
    • 2 yrs ago
    • Reported - view

    Mel,

    Thanks for sharing the email from support. Reindexing might help performance, but it doesn't look like it will do much for file bloat.

    • Sean
    • 2 yrs ago
    • Reported - view

    I'd like to know what this means, "The indexing function should be used by experienced Ninox users". What happens when an inexperienced user clicks the button? 🤔😂

    • PJSpark
    • 2 yrs ago
    • Reported - view

    Just a guess, but maybe the Reorganize function reduces clutter. The database filesize should be a lot less in the reorganized file than the original.

     

    Peter ...

    • Sean
    • 2 yrs ago
    • Reported - view

    Hi Peter,

    Why guess? Have you tested and found that to be the case? I'm not trying to be rude. I just don't have time to test it now. Is that available on the Mac app?

    • Sean
    • 2 yrs ago
    • Reported - view

    Just from memory, I thought "Reorganize" was used to transfer database ownership from one device to another 

    • Mel_Charles
    • 2 yrs ago
    • Reported - view

    Sean

    "The indexing function should be used by experienced Ninox users". Seems like a very patronising statement to me!. Experience is only ever gained by doing/learning something! - Ie Pratice-Feedback-Learn! - The suggestion from Ninox is that it will reduce unwanted space etc by compactation etc.

    With my very large database and weekly uodates of over 30k records at a time I can test this on a copy DB and report back, but it will have a to wait about abouut a week until I get a chance.

Content aside

  • 2 yrs agoLast active
  • 11Replies
  • 899Views