This issue has been resolved. CloudStor and FileSender are now reliably supporting file uploads >1GB. The Cloud Services team apologise for any inconvenience caused.
Posted Feb 18, 2021 - 09:54 AEDT
Monitoring
The Cloud Services team have identified a likely cause of the file upload and performance issue and have applied changes to mitigate this issue. Our internal testing has shown improved performance and completion rates for file uploads >1GB in the browser, using the sync client and using RClone. We will continue to monitor the system performance and will provide further updates next week. We still recommend using RClone as the preferred method to upload files >1GB on a regular basis.
Posted Feb 12, 2021 - 11:53 AEDT
Update
The Cloud Services team are continuing to address the issues with large file uploads and are currently applying configuration tuning to the file system on the advice of our storage partner. The team are treating this with the highest priority and we will provide updates as this work progresses.
Posted Feb 08, 2021 - 15:56 AEDT
Update
The Cloud Services team are working with our software vendors to apply multiple updates to CloudStor to overcome the issues with large file uploads. We understand that the updates applied on 27/01/21 have not provided a significant improvement. We will be scheduling more system updates over the coming weeks. However, our current advice is to avoid using the browser or sync client for file uploads greater than 1 GB. In our testing we see higher success rates for large file uploads using the RClone and Rocket tools, and suggest using these tools outside of business hours. Information on these tools is available via our knowledge base: https://support.aarnet.edu.au/hc/en-us/articles/360000911395-Preferred-Client-Tools
Posted Jan 28, 2021 - 16:44 AEDT
Update
The AARNet Cloud Services team are continuing to work with our software partners to analyse the root cause of these issues. The team have been testing new software to help address these issues and expect to implement this as early as the week beginning January 18, if not earlier.
We apologise for the significant inconvenience caused. We will provide regular updates on progress here.
Posted Jan 11, 2021 - 15:30 AEDT
Update
CloudStor is still experiencing errors for large file uploads. This is generally for files > 1GB but excessive waits and timeouts may occur for smaller files as well. Users may see "Upload Failed" errors in their browser, or "400 Bad Request" errors in client based uploads. The AARNet Cloud Services team are working with our software partners to diagnose the root cause of these issues. We apologise for the significant inconvenience caused. We will provide regular updates on progress here.
Posted Dec 17, 2020 - 16:22 AEDT
Update
The Cloud Services team have identified the cause of the file upload failures for large files. File uploads have improved for the majority of files larger than 1 GB, and we are working towards a complete remediation.
Posted Dec 17, 2020 - 08:57 AEDT
Identified
The Cloud Services team are making changes on the file upload services that we expect will significantly reduce the number of upload failures for >1 GB files. The changes will be implemented this Friday afternoon during the current maintenance period. We will then monitor the uploads and identify next steps early next week. We apologise for the inconvenience caused and thank you for your patience as we work through these issues.
Posted Dec 11, 2020 - 16:28 AEDT
Update
The Cloud Services team are still investigating this issue of an increase of error rate for large file uploads on CloudStor, and are treating it as the highest priority. We apologise for the inconvenience caused and we are working towards a resolution as quickly as possible.
Posted Dec 09, 2020 - 14:35 AEDT
Investigating
The Cloud Services team have identified an increase of error rate for large file uploads on CloudStor. Users will see "Upload Failed" errors in their browser, or "400 Bad Request" errors in client based uploads. This may be resolved by retrying, or smaller uploads. The team are investigating as a matter of priority and we will keep you informed as the investigation progresses.
