Tag Archives: Eagle

Data Management – Modify Eagle/Owl Cloud Sync Account

Re-examining our backup options for our two Synology servers (Eagle & Owl), I realized that they were both backing up to the just my account on UW’s unlimited Google Drive storage.

The desired backup was to go to our shared UW account, so that others in the lab would have access to the backups.

Strangely, I could not add the shared UW account (srlab) to my list of Google accounts. In order to verify the shared UW account with Google, I had to connect to the servers’ web interfaces in a private browsing session and then I was able to provide the correct user account info/permissions.

Anyway, it’s all going to our shared UW account now.

 

SELECT GOOGLE DRIVE AS THE SYNC PROVIDER:

 

 

 

 

SHARED UW ACCOUNT IS NOT A CHOICE:

 

 

TRY “ADD ACCOUNT”:

 

BUT ADD ACCOUNT DOESN’T WORK (DROP-DOWN MENU DOESN’T OFFER SRLAB AS A CHOICE)”

 

 

 

REPEAT STEPS, BUT CONNECT TO SYNOLOGY VIA PRIVATE BROWSING SESSION AND IT’S GOOD TO GO:

 

 

SET LOCAL AND REMOTE FOLDERS:

 

 

CONFIRMATION THAT IT’S SET UP:

 

 

AND, IT’S RUNNING:

 

Data Management – Synology Cloud Sync to UW Google Drive

After a bit of a scare this weekend (Synology DX513 expansion unit no longer detected two drives after a system update – resolved by powering down the system and rebooting it), we revisited our approach for backing up data.

Our decision was to utilize UW Google Drive, as it provides unlimited storage space!

Synology has an available app for syncing data to/from Google Drive, so I set up both Owl (Synology DS1812+) and Eagle (Synology DS413) to sync all of their data to a shared UW Google Drive account. This should provide a functional backup solution for the massive amounts of data we’re storing and it will simplify tracking where/what is backed up where. Now, instead of wondering if certain directories are backed up via CrashPlan or Backblaze or Time Backup to another Synology server, we know that everything is backed up to Google Drive.

 

 

 

 

 

Data Analysis – Identification of duplicate files on Eagle

Recently, we’ve been bumping into our storage limit on Eagle (our Synology DS413):

 

Being fairly certain that there’s a significant amount of large datasets that is duplicated throughout Eagle, I ran a program on Linux called “fslint”. It searches for duplicates files based on a few parameters and is smart enough to be able to compare files with different filenames that share the same file contents!

I decided to check for duplicate files in the Eagle/archive folder and the Eagle/web folder. Initially, I tried searching for duplicates across all of Eagle, but after a week of running I got tired of waiting for results and ran the analysis on those two directories independently. As such, there is a possibility that there are more duplicates (consuming even more space) across the remainder of Eagle that have not been identified. However, this is a good starting point.

Here are the two output files from the fslint analysis:

 

To get a summary of the fslint output, I tallied the total amount of duplicates files that were >100MB in size. This was performed in a Jupyter notebook (see below):
Notebook Viewer: 20160114_wasted_space_synologies.ipynb
Jupyter (IPython) Notebook File: 20160114_wasted_space_synologies.ipynb

 

Here are the cleaned output files from the fslint analysis:

 

Summary

There are duplicates of files (>100MB in size) that are consuming at least 730GB!

Since the majority of these files exist in the Eagle/web folder, careful consideration will have to be taken in determining which duplicates (if any) can be deleted since it’s highly possible that there are notebooks that link to some of the files. Regardless, this analysis shows just how space is being consumed by the presence of large, duplicate files; something to consider for future data handling/storage/analysis with Eagle.

Uninterruptible Power Supplies (UPS)

A new UPS we installed this week for our qPCR machine (Opticon2 – BioRad) to handle power surges and power outages doesn’t seem to be working properly. With the qPCR machine (and computer and NanoDrop1000) plugged into the “battery” outlets on the UPS, this is what happens when the Opticon goes through a heating cycle:

The UPS becomes overloaded when the Opticon is in a heating cycle.

 

And, sometimes, that results in triggering a fault, shutting everything off in the middle of a qPCR run:

Fault message indicating unit overload.

 

This is supremely lame because having a battery backup is a great way to prevent the qPCR machine from shutting off when a power outage occurs!

 

I switched the Opticon (and computer and NanoDrop1000) to the outlets that are solely for surge protection. Check out what happens when I run the qPCR machine now:

Opticon plugged in to surge protection outlet while in heating cycle. Notice that output load is 0%.

 

So, I guess we’ll settle for at least having the surge protection aspect of things.

 

While handling this UPS issue, I realized that the two Synology servers we have possess a built-in UPS monitor. So, I connected the USB cables to/from each of the UPS that each server is plugged into and enabled UPS shutdown in the Synology Diskstation Management (DSM):

 

Eagle

 

Owl

 

Now, both Synology units will enter Safe Mode when the UPS they’re connected to reaches a low battery status. This will help minimize data loss/corruption during the next extended power outage we experience.

Server Email Notifications Fix – Eagle

The system was previously set to use Steven’s Comcast SMTP server. Sending a test email from Eagle failed, indicating authentication failure. I changed this to use the University of Washington’s email server for outgoing messages. Here’s how…

In the Synology Disk Station Manager (DSM):

Control Panel > Notifications

  • Service provider: Custom SMTP Server
  • SMTP server: smtp.washington.edu
  • SMTP port: 587
  • Username: myUWnetID@uw.edu
  • Password: myUWpassword