Received notice the sequencing data was ready from Genewiz for the samples submitted 20151222.
Download the FASTQ files from Genewiz project directory:
wget -r -np -nc -A "*.gz" ftp://username:password@ftp2.genewiz.com/Project_BS1512183
Since two species were sequenced (C.gigas & O.lurida), the corresponding files are in the following locations:
http://owl.fish.washington.edu/nightingales/O_lurida/
http://owl.fish.washington.edu/nightingales/C_gigas/
In order to process the files, I needed to identify just the FASTQ files from this project and save the list of files to a bash variable called ‘bsseq':
bsseq=$(ls | grep '^[0-9]\{1\}_*' | grep -v "2bRAD")
Explanation:
bsseq=
- This initializes a variable called “bsseq” to the values contained in the command following the equals sign.
$(ls | grep '^[0-9]\{1\}_*' | grep -v "2bRAD")
- This lists (ls) all files, pipes them to the grep command (|), grep finds those files that begin with (^) one or two digits followed by an underscore ([0-9{1}_*), pipes those results (|) to another grep command which excludes (-v) any results containing the text “2bRAD”.
| FILENAME |
SAMPLE NAME |
SPECIES |
| 1_ATCACG_L001_R1_001.fastq.gz |
1NF11 |
O.lurida |
| 2_CGATGT_L001_R1_001.fastq.gz |
1NF15 |
O.lurida |
| 3_TTAGGC_L001_R1_001.fastq.gz |
1NF16 |
O.lurida |
| 4_TGACCA_L001_R1_001.fastq.gz |
1NF17 |
O.lurida |
| 5_ACAGTG_L001_R1_001.fastq.gz |
2NF5 |
O.lurida |
| 6_GCCAAT_L001_R1_001.fastq.gz |
2NF6 |
O.lurida |
| 7_CAGATC_L001_R1_001.fastq.gz |
2NF7 |
O.lurida |
| 8_ACTTGA_L001_R1_001.fastq.gz |
2NF8 |
O.lurida |
| 9_GATCAG_L001_R1_001.fastq.gz |
M2 |
C.gigas |
| 10_TAGCTT_L001_R1_001.fastq.gz |
M3 |
C.gigas |
| 11_GGCTAC_L001_R1_001.fastq.gz |
NF2_6 |
O.lurida |
| 12_CTTGTA_L001_R1_001.fastq.gz |
NF_18 |
O.lurida |
I wanted to add some information about the project to the readme file, like total number of sequencing reads generated and the number of reads in each FASTQ file.
Here’s how to count the total of all reads generated in this project
totalreads=0; for i in $bsseq; do linecount=`gunzip -c "$i" | wc -l`; readcount=$((linecount/4)); totalreads=$((readcount+totalreads)); done; echo $totalreads
Total reads = 138,530,448
C.gigas reads: 22,249,631
O.lurida reads: 116,280,817
Code explanation:
totalreads=0;
- Creates variable called “totalreads” and initializes value to 0.
for i in $bsseq;
- Initiates a for loop to process the list of files stored in $bsseq variable. The FASTQ files have been compressed with gzip and end with the .gz extension.
do linecount=
- Creates variable called “linecount” that stores the results of the following command:
`gunzip -c "$i" | wc -l`;
- Unzips the files ($i) to stdout (-c) instead of actually uncompressing them. This is piped to the word count command, with the line flag (wc -l) to count the number of lines in the files.
readcount=$((linecount/4));
- Divides the value stored in linecount by 4. This is because an entry for a single Illumina read comprises four lines. This value is stored in the “readcount” variable.
totalreads=$((readcount+totalreads));
- Adds the readcount for the current file and adds the value to totalreads.
done;
echo $totalreads
- Prints the value of totalreads to the screen.
Next, I wanted to generate list of the FASTQ files and corresponding read counts, and append this information to the readme file.
for i in $bsseq; do linecount=`gunzip -c "$i" | wc -l`; readcount=$(($linecount/4)); printf "%s\t%s\n%s\t\t\n" "$i" "$readcount" >> readme.md; done
Code explanation:
for i in $bsseq; do linecount=`gunzip -c "$i" | wc -l`; readcount=$(($linecount/4));
- Same for loop as above that calculates the number of reads in each FASTQ file.
printf "%s\t%s\n\n" "$i" "$readcount" >> readme.md;
- This formats the the printed output. The “%s\t%s\n\n” portion prints the value in $i as a string (%s), followed by a tab (\t), followed by the value in $readcount as a string (%s), followed by two consecutive newlines (\n\n) to provide an empty line between the entries. See the readme file linked above to see how the output looks.
>> readme.md; done
- This appends the result from each loop to the readme.md file and ends the for loop (done).
