When you upload data into Starter the system does much more than copy it from your device into cloud storage and display it, please read on for a rundown of all the steps that go on behind the scenes:
- The first step is the actual upload to get the files from your local device up into a storage area in the Amazon AWS cloud. The time this step will take will vary dependent on the speed of your internet connection.
- Next is a check of the file’s integrity and generation of a fixity checksum (more about that below).
- The next thing the system does it to check whether the file is a duplicate of any other file currently stored in your system. This is not based on filename as that is not a unique attribute, but rather on the checksum created in the previous step, this is generated against the bit level content of the file. This checksum is compared to existing ones in the system for a match. It is entirely reasonable to have true duplicates in the system and they can happily co-exist, one thing we are looking to do in the future is provide a way of reporting on any duplicates and allow you to decide what action to take with them.
- A virus check is then carried out to ensure that you are not storing anything untoward along with your files.
- The file goes through a characterisation process to ascertain the file format. Again this is not based on the filename or extension but on bit level information to compare the contents of the file against a known signature. The characterisation process also extracts all the technical metadata for display in the Advanced Information tab.
- The file is then placed in the structure and permanently stored in the system, at this stage the file is now preserved.
- Finally preservation actions are carried out, these include creating access copies and thumbnails where applicable. These actions are carried out in the background so you will be able to see your new content before these are complete.