CONSED 15.0 DOCUMENTATION CONTENTS: WHAT IS NEW IN CONSED 15.0 INSTALLING CONSED<--------------------- QUICK TOUR OF CONSED WHAT IS AUTOFINISH USING AUTOFINISH USING AUTOMATED ADD NEW READS USING AUTOPCRAMPLIFY USING AUTOEDIT ADVANCED PHRAP/CONSED USAGE NOTE TO LINUX USERS NOTE TO ITANIUM LINUX USERS NOTE TO SGI USERS NOTE TO SOLARIS USERS NOTE TO MACOSX USERS CONSED CUSTOMIZATION CONSED FOR LARGE ASSEMBLIES FOR PROGRAMMERS AND FELLOW TRAVELLERS ONLY MONITORS AND MICE FOR CONSED AUTOFINISH AND PRIMER-PICKING PARAMETERS ACE FILE FORMAT WHAT THE COLORS MEAN ---------------------------------------------------------------------------- WHAT IS NEW IN CONSED 15.0 AMD 64 Bit Linux Now Available Linux version more stable There are actually many different versions of linux, and incompatibilities between consed and some of them caused problems. I believe all of these problems are solved. For Rapid Review of Many Different Projects Can write a script that starts up consed repeatedly on different projects, each time with a custom navigation file loaded consed -ace (ace file) -nav (custom navigation file) Assembly View Shows Restriction Fragments (and shows which ones don't match the gel). Can find a contig by typing its name--useful for projects with hundreds or thousands of contigs Traces Window Shows Consensus Tags Autofinish improvements in PCR Autofinish and Assembly View Can import PCAP contig ordering information List of Contigs in Main Window Can be reordered by # of reads AutoReport feature Print out assembly information, such as the list of scaffolds (how the contigs are ordered and oriented) Show All Traces vertical scrolling E.g., can move by exactly one screenful of traces Many bugs fixed ---------------------------------------------------------------------------- INSTALLING CONSED You MUST have the following phred, phrap, phd2fasta, and crossmatch in order to use this version of Consed: 000925.c or later for phred 0.990319 or later for phrap and crossmatch 0.990622.e or later for phd2fasta (supplied with this version of consed) any version of addReads2Consed.perl (supplied with this version of consed) 030415 of phredPhrap (supplied with this version of consed) (Note: if you have an older version of phredPhrap, some of the more recent Consed features, such as miniassemblies, will not work. Note to existing polyphred users: phredPhrap now calls polyphred with different parameters which will give cause it to apply different tags than it used to, but these different tags will give it behavior consistent with that described below in CONSED-POLYPHRED INTERACTION. For more information, see http://droog.gs.washington.edu/PolyPhred.html ) 030117 or later for transferConsensusTags.perl (supplied with this version of consed) any version of tagRepeats.perl (supplied with this version of consed) any version of determineReadTypes.perl (or your own custom modified version) For phred, contact bge@u.washington.edu (Brent Ewing) For phrap and crossmatch, contact phg@u.washington.edu (Phil Green) Summary of files your must edit (instructions are below): addReads2Consed.perl determineReadTypes.perl phredPhrap primerCloneScreen.seq primerSubcloneScreen.seq repeats.fasta vector.seq In order to run the gauntlet of phred/phd2fasta/crossmatch/phrap, there is a perl script phredPhrap supplied with Consed (above). YOU MUST USE THIS PERL SCRIPT. If you try to run each of these programs directly, you are on your own and you will probably spend a lot of time needlessly. 1) After downloading the distribution with netscape (see www.phrap.org and click on 'Consed'), copy the distribution to a unix computer (if it is not already on one). Unpack the files by typing the appropriate line below (which one depends on what you named the file downloaded by netscape): zcat consed_linux.tar.Z | tar -xvf - zcat consed_linux_itanium.tar.Z | tar -xvf - zcat consed_solaris.tar.Z | tar -xvf - zcat consed_macosx.tar.Z | tar -xvf - zcat consed_amd64.tar.Z | tar -xvf - zcat consed_alpha.tar.Z | tar -xvf - zcat consed_sgi.tar.Z | tar -xvf - zcat consed_ibm.tar.Z | tar -xvf - zcat consed_solaris_intel.tar.Z | tar -xvf - zcat consed_hp.tar.Z | tar -xvf - Note: You must run tar on a UNIX computer--not on an Windows computer, due to a difference in the handling of breaks between lines. 2) I suggest you put Consed, phred, crossmatch, phrap, the perl scripts, and other executables into /usr/local/genome/bin. So create /usr/local/genome/bin and /usr/local/genome/lib If you can't actually use /usr/local/genome, then you could make /usr/local/genome be a link to the real location--that will work just as well. If you want to have another location xxx, then put: setenv CONSED_HOME xxx into the .cshrc (or equivalent if you are using bash or a shell other than csh or tcsh) of all Consed users and create $CONSED_HOME/bin and $CONSED_HOME/lib and put all of these programs into $CONSED_HOME/bin 3) Make sure that /usr/local/genome/bin (or $CONSED_HOME/bin) is in every Consed users' PATH. 4) Put the Consed executable in /usr/local/genome/bin (or $CONSED_HOME/bin) Read the appropriate section of this document: NOTE TO SOLARIS USERS, NOTE TO SGI USERS, NOTE TO MACOSX USERS, NOTE TO LINUX USERS, or (if you are running Linux on an Itanium--a big 64 bit box) NOTE TO ITANIUM LINUX USERS. 5) Check this by logging on as a user and typing: (consed executable name) -V where (consed_executalbe name) is one of: consed_linux2.4 consed_linux2.6 consed_linux2.6_dyn consed_solaris consed_mac consed_linux_itanium consed_hp consed_solaris_64 consed_amd64 consed_ibm consed_alpha consed_solaris_intel consed_sgi You should see 'Version 15.0'. If you see something else, you have some debugging to do. 6) TESTING CONSED Follow the first few steps of USING CONSED GRAPHICALLY of the QUICK TOUR (below). If you have problems, it may be due to your X emulator. See 'MONITORS AND MICE FOR CONSED' below. If you get some error such as: Error: Can't open display: then the problem may have nothing to do with Consed, but rather with X. To test this, run some other X application (such as xclock, xterm, xeyes, or xcalc) and see if you get the same error. 7) Build phd2fasta: Go to the misc/phd2fasta directory and type 'make' Move the phd2fasta executable to /usr/local/genome/bin (or $CONSED_HOME/bin) 8) Build mktrace: Go to the misc/mktrace directory and type 'make' Move the mktrace executable to /usr/local/genome/bin (or $CONSED_HOME/bin) 9) Move all perl scripts from the scripts directory to /usr/local/genome/bin (or $CONSED_HOME/bin) Make sure all are executable (chmod a+x *) 10) Get perl 5. (If you have Linux, you already have it so you can skip this step.) You can check where to get perl via the perl web site: http://www.perl.com (If you don't know about perl, try it--it will save you a huge amount of time over developing the same utilities in C, awk, or csh or sh.) Regardless where you put perl, put a link to it in /usr/bin so that all of the scripts with #!/usr/bin/perl will work and you won't have to edit all of them everytime a new Consed release comes out. 11) Create a subdirectory /usr/local/genome/lib/screenLibs. (If you are using a location other than /usr/local/genome for the root of all Phred/Phrap/Consed programs, create $CONSED_HOME/lib/screenLibs). From the misc subdirectory, copy primerCloneScreen.seq and primerSubcloneScreen.seq to the directory /usr/local/genome/lib/screenLibs (or $CONSED_HOME/lib/screenLibs). primerCloneScreen.seq is used to screen candidate primers when you use Consed's function "Pick Primer from Clone Template" (on the Aligned Reads Window). primerSubcloneScreen.seq is used to screen candidate primers when you use Consed's function "Pick Primer from Subclone Template" (on the Aligned Reads Window). Take a look at these files. They are dummy files indicating the fasta format of the sequences that should be put in them. You should put into primerCloneScreen.seq the vector sequence of the cloning vectors you are using (BAC or cosmid) and into primerSubcloneScreen.seq the sequencing vectors you are using (plasmid, M13, etc). Don't be too generous in putting lots of vectors into the files! The larger they are, the slower primer picking will be. Our files are only this big: -rw-r--r-- 1 root root 29938 Nov 7 1997 primerCloneScreen.seq -rw-r--r-- 1 root root 7381 Aug 13 1997 primerSubcloneScreen.seq and primer picking is quite fast enough. Now that you have set this up, you should try the PRIMER PICKING sections in the Quick Tour (above) to make sure this works. 12) You should create a file /usr/local/genome/lib/screenLibs/vector.seq (or $CONSED_HOME/lib/screenLibs/vector.seq if you are not using /usr/local/genome for the root of the Phred/Phrap/Consed files.) This contains all the vector sequences (in FASTA format) that you want to mask out before running phrap. In general, it is the combination of primerCloneScreen.seq and primerSubcloneScreen.seq. 13) You should create a file /usr/local/genome/lib/screenLibs/repeats.fasta (or $CONSED_HOME/lib/screenLibs/repeats.fasta if you are not using /usr/local/genome for the root of the Phred/Phrap/Consed files.) In this file, put any sequences (in FASTA format) that you want to have automatically tagged. These typically are ALU sequences. If you don't want to tag anything, then comment out (put '#' as the first character of the line) the following lines in phredPhrap: Change: !system( "$tagRepeats $szAceFileToBeProduced" ) || die "some problem running $tagRepeats"; to: #!system( "$tagRepeats $szAceFileToBeProduced" ) # || die "some problem running $tagRepeats"; 14) You should create a file /usr/local/genome/lib/screenLibs/singleVectorForRestrictionDigest.fasta containing the cloning vector sequence. This is used for doing in-silico restriction digests. Thus this cloning vector must start at precisely the site where you cut the vector to ligate the insert. It is not sufficient to just download the vector sequence from Genbank. You might need to have it start in a different place. 15) SETTING UP TEST DIRECTORIES Copy the test directories and their contents to some location where the users have write access. Copy--do not move them--because the users will occasionally want a fresh copy. cp -r standard new_location cp -r autofinish new_location cp -r assembly_view new_location cp -r polyphred new_location cd new_location chmod -R a+w * 16) MODIFYING determineReadTypes.perl Read the comments in determineReadTypes.perl Phrap, Consed's primer picking, and Consed/Autofinish all need the following information for each read: is it a univeral primer forward, a universal primer reverse, or a walking read? what is its template name? If you are using different libraries that have different insert sizes, then Consed/Autofinish also need the library name for each read. Generally this information can be determined from the read name, using *your* naming convention. Modify the perl script determineReadTypes.perl to put this information at the end of the phd file using WR info items. If you don't want to do much perl programming and all your libraries have the same insert size, you have the option of using the St Louis naming convention. In this case, the only perl programming you need do is to comment out (put a "#" in front) the line in determineReadTypes.perl that starts with: die "You must edit determineReadTypes.perl You must also uncomment (remove the "#"s in column 1) the lines in the phredPhrap script that say roughly: #print "\n\n--------------------------------------------------------\n"; #print "Now running determineReadTypes.perl...\n"; #print "--------------------------------------------------------\n\n\n"; #!system( "$determineReadTypes" ) || die "some problem running determineReadTypes.perl $!\n"; But what is the St Louis naming convention? Most of it (but not all) is explaned in the phrap documentation. In addition, you must never use an underscore in the name if the read is a universal primer forward or universal primer reverse read. If the read is a walk, then you must have an underscore (_) follow the template name and then have a number (the oligo number). Examples of reads in the St Louis naming convention: read eeq03a01.g1.phd.1 is univ rev template: eeq03a01 library: eeq03 read eeq03a02.b1.phd.1 is univ fwd template: eeq03a02 library: eeq03 read eeq03a02.g1.phd.1 is univ rev template: eeq03a02 library: eeq03 read eeq03a03.b1.phd.1 is univ fwd template: eeq03a03 library: eeq03 read eej45h07_2.i1.phd.1 is walk template: eej45h07 library: eej45 read eej46c12_1.i1.phd.1 is walk template: eej46c12 library: eej46 Once you have correctly customized determineReadTypes.perl, then uncomment the line in phredPhrap which calls determineReadTypes.perl It is fine to assume the St Louis naming convention for the purpose of the sample dataset directories that come with Consed ("standard", "assembly_view", "autofinish", and "polyphred"). 17) TROUBLESHOOTING YOUR CHANGES TO determineReadTypes.perl Consed allows you to check that you have correctly modified determineReadTypes.perl: On the Consed Main Window, point to 'Info', hold down the left mouse button, and release on 'Show Info for Each Read'. Study all the information and check that the information presented is correct. If, for example, Consed thinks that there are templates that have 9 or more reads, it is likely that you have not correctly customized determineReadTypes.perl You will see a section that looks like this: template djs736a2_fp04q286 with 2 reads djs736a2_fp04q286.x2 term universal forward (from phd file) djs736a2_fp04q286.y2 term universal reverse (from phd file) You want to see the "from phd file" part. If, instead of "from phd file", it says "inferred from name", that means that determineReadTypes.perl couldn't figure out what kind of read it was. If you think you have made a mistake in customizing determineReadTypes.perl, it is best to delete the PHD files (and phd.ball if you are using that) and run phredPhrap again since the otherwise incorrect WR items will be left in the PHD files. There is more specific documentation within the script determineReadTypes.perl for more information about how to customize it. CUSTOMIZING determineReadTypes.perl: SPECIAL CASES 18) FAKE READS By "fake reads" I mean reads such as those created from a Genbank reference sequence or a consensus from some other assembly... or others for which there is no chromatogram (and there never was any chromatogram). If you don't use any such reads, you can skip this step. In the past, any read that ended with a .a2 or .c3 (where 2 and 3 could be any numbers), was considered a fake read. Now you can make Autofinish not assume this using the .consedrc parameter (see CONSED CUSTOMIZATION): consed.fakeReadsSpecifiedByFilenameExtension: false Instead, you must have determineReadTypes.perl put "fake" into the "type:" field of a "template" WR item. See determineReadTypes.perl for more information. After installing Consed, you should run all the following tests to make sure you have installed everything correctly: 19) APPENDING EXPID TO THE PHD FILES If you are using Autofinish, and would like Autofinish to tell you how well your reads are succeeding, then the phd files must be appended with the experiment id's. In the 3 Autofinish summary files (*.univReverse, *.univForwards, and *.customPrimers), you will see information like this: univ rev,,,->,-329,-249,71,Contig1,3,djs228_1034 or this: tgaagaaatggctgactcc,56,1,->,3258,3338,3658,Contig1,4,djs228_2813,5,djs228_168,6,djs228_1248 The '3' just before the djs228_1034 on the line starting with "univ rev" is an experiment id. There is also an expid '4' just before djs228_2813, an expid '5' before djs228_168, and an expid '6' just before djs228_1248. Autofinish doesn't know what you will end up calling these reads it is telling you to make. Autofinish only knows those reads by the numbers 3, 4, 5, and 6. So when you make the reads, Autofinish needs to be informed that this is 'experiment 3' or whatever. You do this by appending in the phd file the following structure: WR{ expid addExpid 990811:140818 5 } where WR stands for 'whole read item', expid for 'expid' addExpid is the name of the program that you will write that will append this information 990811:140818 is the date and time in format YYMMDD:HHMISS 5 is the expid This program must be run *after* phred runs to create the phd files. Thus your program must have some method of determining what the expid of each read is. What the University of Washington Genome Center does is to have the finishers put the expid as part of the filename. This makes it easy for a program to look at the phd file and figure out what the expid is and then write the WR item into that phd file. Alternatively, you could keep a database and, after the phd file is created, look into the database to see what the expid is. When you have successfully added expid's to the phd files, the next time you run Autofinish on this project, it will have in the 'EVALUATE' section of the Autofinish output file, lots of interesting information about how well the reads succeeded. TESTING The following tests must be done to insure that the installation was done correctly. 20) TESTING RESTRICTION DIGEST Try the restriction digest feature (RESTRICTION DIGEST above) to make sure this works. 21) TESTING ADD NEW READS It will make your life easier if phred, phrap, and crossmatch are all where Consed expects them: in /usr/local/genome/bin 22) Decide where to put phred's parameter file phredpar.dat and edit both addReads2Consed.perl and phredPhrap to reflect this location. I generally prefer to put it in /usr/local/genome/lib to keep all of the Phred/Phrap/Consed files in one place. Alternatively, you could put it in /usr/local/etc/PhredPar/phredpar.dat which is the historical location of this file. 23) Next you should test the ADD NEW READS step in the Quick Tour (below). This step requires that everything be set up correctly and in the correct location. Hopefully the error messages are clear enough to help you if you have set up anything incorrectly. 24) TESTING RUNNING CROSSMATCH FROM ASSEMBLY VIEW See RUNNING CROSSMATCH FOR SEQUENCE MATCHES (above) and make sure that step works. 25) TEST RUNNING PHREDPHRAP See the section RUNNING PHRED and PHRAP in the Quick Tour (below) 26) TESTING MINIASSEMBLIES See PULLING OUT READS AND RE-ASSEMBLYING THEM (MINIASSEMBLIES) and MINIASSEMBLIES (below) and make sure those steps work. The newer version of phredPhrap is required for this. If you have invested a lot of work customizing some old version of phredPhrap, and don't want to upgrade, you do have the option of keeping your customized version of phredPhrap for regular assemblies, and using the new version of phredPhrap for miniassemblies. To do this, you must specify the alternate name/location of phredPhrap by the .consedrc parameter: consed.fullPathnameOfMiniassemblyScript: /usr/local/genome/bin/phredPhrap (See CONSED CUSTOMIZATION below.) USING YOUR OWN DATA 27) Create the following directory structure, which can be anywhere on any disk: Directory structure: top level directory (generally named after the BAC or cosmid) subdirectory 'chromat_dir'--chromatograms go in here subdirectory 'phd_dir'--phd files will automatically be put here subdirectory 'edit_dir'--ace files will automatically be put here Put the chromatogram files (e.g., .ab1 or .scf files) into the chromat_dir directory. Keep phd_dir and edit_dir empty. If you already have your chromatograms somewhere else, you can make chromat_dir be a link to wherever you have them. The various phrap and crossmatch files will be put into edit_dir by the phredPhrap script. 28) cd to the edit_dir directory, and type: phredPhrap If you are successful, the script will tell you so. (You can also look in phd_dir and you will see phd files for each of the chromatograms you added in chromat_dir. If the phd files are missing, then phred was unable to call bases from the chromatograms in chromat_dir and you will need to figure out why not). Make sure you are in edit_dir and bring up Consed on the ace file: 29) Bring up Consed as shown in the "QUICK TOUR OF CONSED" consed You should see a file with the extension .ace.1 Double click on it. You should see a list of contigs. Double click on the one you want to see. Follow the first few steps of the QUICK TOUR OF CONSED (below). You should at least go as far as viewing traces. ---------------------------------------------------------------------------- QUICK TOUR OF CONSED Release 15.0 Consed is a program for viewing and editing assemblies assembled with the phrap assembly program. If you are already an advanced Consed user, you should read through this and do any of the exercises on features that you are unfamiliar with. I frequently run across people who are doing something in Consed a hard way month after month, and request a new feature to make things easier, when that new feature is already in Consed. If you have never used Consed before, to follow this Quick Tour will take you less than 6 hours. However, it will save you approximately 2 days in agony. If you have 2 extra days to spare, and prefer to waste them in agony, then do not do this Quick Tour and instead immediately skip down to 'INSTALLING CONSED' above. If you do the Quick Tour, start your system administrator installing consed (see INSTALLING CONSED (above)) because you will need to have completed that for some of the more advanced sections of the Quick Tour. When you do the quick tour, I encourage you to be free about changing the data set. If you really mess things up (such as changing all a read's bases to N's), no problem--just delete the data set and start again with a fresh copy. USING CONSED GRAPHICALLY 30) Type the following: cd standard/edit_dir 31) start Consed by typing the appropriate command below: ../../consed_solaris ../../consed_solaris64 ../../consed_alpha ../../consed_hp ../../consed_sgi ../../consed_linux2.4 ../../consed_linux2.6 ../../consed_linux2.6_dyn ../../consed_mac ../../consed_linux_itanium ../../consed_solaris_intel ../../consed_ibm ../../consed_amd64 ../../consed_amd64_dyn (Don't worry about a message like: Warning: Cannot convert string "helvetica" to type FontStruct ) Two windows will appear. One of these will have the list of .ace files and say 'select assembly file to open' and 'standard.fasta.screen.ace.1'. Double click on "standard.fasta.screen.ace.1". The first window goes away. You will now see a list of one contig and a list of reads. This is the 'Consed Main Window'. Double click on 'Contig1'. The 'Aligned Reads Window' will appear. 32) SCROLLING Try scrolling back and forth. Try scrolling by dragging the thumb of the scrollbar. Also try scrolling by clicking on the 4 buttons: << < > >> for scrolling by small amounts. For scrolling by tiny amounts, click on the arrows at either end of the scrollbar. For scrolling by huge amounts, use the middle mouse button and just click on some location on the scrollbar. For scrolling to the beginning or end of the contig, use the <<< or >>> buttons. (Question: why can't you just move the scrollbar to the extreme right in order to go to the end of the contig? Answer: in typical assemblies, there are reads that protrude beyond the beginning of the contig and reads that protrude beyond the end of the contig. Moving the scrollbar to the extreme right will scroll the contig to the end of the rightmost read--typically far to the right of the end of the contig. Thus you should get in the habit of using the <<< and >>> buttons.) GOTO POSITION 33) In the Aligned Reads Window, click in the 'Pos:' box in the upper right-hand corner. Type in a number, such as 540, and push the 'Return' or 'Enter' key. The Aligned Reads Window will scroll to position 540. We find this feature is particularly useful when one person wants another person to look at something in the sequence. 34) COLORS Notice the colors. Scroll to position 937 and notice the read 'a'. The red bases are the ones that disagree with the consensus. Notice the different shades of grey background (around the bases). They have the following meanings, but first, you need to understand the meaning of the quality values: A quality value of 10 means 1 error in ten to the 1.0 power A quality value of 20 means 1 error in ten to the 2.0 power A quality value of 30 means 1 error in ten to the 3.0 power A quality value of 40 means 1 error in ten to the 4.0 power and for quality values in between: A quality value of 25 means 1 error in ten to the 2.5 power Get the idea? (These have actually been empirically verified--if you are interested in the gory details, read the phred papers: Ewing B, Hillier L, Wendl M, Green P: Basecalling of automated sequencer traces using phred. I. Accuracy assessment. Genome Research 8, 175-185 (1998). Ewing B, Green P: Basecalling of automated sequencer traces using phred. II. Error probabilities. Genome Research 8, 186-194 (1998). In that same copy of the journal is a paper about Consed, as well.) Also notice the upper and lowercase. This is just a cruder indication of the quality of the bases. 35) To see the quality value of a particular base, point at it and click with the left mouse button. You will see the quality displayed in the Info Box at the bottom of the Aligned Reads Window. These quality values are shown in grey scales: Quality 0 through 4 is given by dark grey Quality 5 through 9 is given by a shade lighter Quality 10 through 14 is given by a shade still lighter . . . Quality of 40 through 97 is given by white (the brightest shade) A quality value of 99 is reserved for bases that have been edited and the user is absolutely sure of the base ('high quality edited'). A quality value of 98 is reserved for bases that have been edited and the user is not sure of the base ('low quality edit'). The ends of the reads shows bases that are grey and have a black background. These are the low quality ends of the reads or the unaligned ends of reads, as determined by phrap. 36) Click on a base on a read. Then hold down the control key and type 'a'. You will move to the beginning of the read. Hold down the control key and type 'e'. You will move to the end of the read. (Emacs users will recognize these commands.) 37) HIGHLIGHTING READ NAMES In the Aligned Reads Window, click on a read name with the left mouse button. The name will turn magenta. Click again and it will turn yellow again. Try turning it magenta and then scrolling. This feature is helpful in keeping track of a particular read as you scroll. If you have an emacs window open (or any editor window), you can paste the read name in by just clicking with the middle mouse button. When you clicked on the read name in the Aligned Reads Window with the left mouse button, the read name was loaded into the paste buffer. 38) DIMMING ENDS OF READS Scroll so that location 490 is about in the middle of the aligned reads window. Push the left mouse button down on the menu item 'Dim'. There will be a list of choices that will appear. Drag the cursor down to 'Dim Nothing' and release. Now look what happened to the color of the bases. The ends of the reads that used to be with a black background now appear red with a grey background. You are seeing the clipped-off bases with all the same information as any other base. Since there is a huge amount of red (discrepant) bases, the screen becomes distracting and busy. Thus by default the low quality clipped-off bases are made with a black background and a grey foreground so they don't distract you. Notice there is a distinction here between 'low quality ends of reads' and 'unaligned ends of reads'. Unaligned ends of reads can be low quality as well, or they can be high quality, as in the case of chimeric reads. Point with the mouse to a read name and hold down the right mouse button. You will notice there is a line that says "high quality from nnn to nnn; aligned from nnn to nnn; chem: prim". This is giving the same information in number form. Highlight the read name first (see HIGHLIGHTING READ NAMES above) so you don't lose the read as you scroll. Then check that the numbers agree with the dimming. You can play with the dimming options a bit. Then return it to 'Dim Low Quality' for the rest of this tour. TRACES AND EDITING 39) Point with the mouse at a base of one of the reads and click with the middle mouse button. (If you have a 2 button mouse, see MONITORS AND MICE FOR CONSED below.) The Trace Window showing the traces for that stretch of read should popup. There are 2 rows of numbers: 'con' are the consensus positions 'rd' are the read positions There are 3 rows of bases in the trace window: 'con' is the consensus 'edt' is where you can edit the base calls of the read 'phd' is the original phred base calls Notice that a red rectangle blinks (the 'cursor') in the corresponding positions of the Aligned Reads Window and the Trace Window. 40) Try editing in the Trace Window. You can click the left mouse button on a base in the 'edt' line to set the cursor (a blinking red rectangle). You can directly overstrike a base by typing a letter. Try this. Try undoing it (by clicking on 'undo' ). If you want to undo more than one edit, you will have to go back to the main Consed window and click on the button labeled 'Undo Edit...'--you will learn that later. You can overstrike with the following characters: acgt (bases), * (a pad, in effect deleting the base), and mrwsykvhdb (IUB ambiguity codes). You can move left and right with the arrow keys. We believe that the user should change a base call only while examining the traces. That is why editing is done here--not in the Aligned Reads Window. 41) You can insert a column of pads by pushing the space bar. Try this. (You may need to click on a base on the 'edt' line first.) (For those of you new to editing assemblies, a 'pad', which in Consed and phrap is represented by the '*' character, is used to align two or more sequences such as these: gttgacagtaatcta gttgacataatcta in which one sequence has an inserted or deleted base with respect to the other. By inserting the pad character, it is possible to get a good alignment: gttgacagtaatcta gttgaca*taatcta This is the purpose of pad character--it is just a placeholder.) You can then overstrike a pad with a base. In this way you can insert a base, and still preserve the alignment. 42) Try highlighting a stretch of a read on the edt line by holding down the middle mouse button and dragging the cursor over some bases. They will turn yellow as you drag. Then release the mouse button. A window will pop up giving you some choices of what to do with those (yellow) bases.: Make High Quality--makes the highlighted bases edited high quality (99). This tells phrap (when it reassembles) that you are sure of the sequence here. Change Consensus--make the highlighted bases edited high quality and change the consensus to agree with that stretch of the read. This is a directive to phrap (upon reassembly) to use that stretch of that read to be the consensus. Make low quality--makes the highlighted bases edited low quality. This tells phrap (when it reassembles) that you are not sure of the bases here and phrap can go ahead and make a join even if the bases in this region don't match perfectly. Make Low Quality to Left End--same as above, but all the way to the left end of the read. Make Low Quality to Right End--same as above, but all the way to the right end of the read. Change to n's--Change the highlighted bases to n's which means they are unknown bases. This tells phrap (when it reassembles) to not make any join based on these bases. It is useful when you believe the bases may be in the chimeric portion of a read. Change to n's to left--same as above but to left end. Change to n's to right--same as above but to right end. Change to x's to left--Change the highlighted bases to x's which means they are vector. This tells phrap to ignore these bases for the purpose of determining overlap. Change to x's to right--same as above but to right end. Add Tag--allows user to add any tag to a stretch of read bases. Dismiss--you decided you don't really want to do anything with this stretch of bases. This popup is made so that nothing else works until you choose something. Try each of these choices, except for tags, which you'll try below. 'Change Consensus' has an additional function--if a read extends out on the right beyond the end of the consensus, you can extend the consensus by using this function. You might want to do this, for example, if crossmatch did not correctly find the cloning site and thus clipped too much. You can add these bases to the consensus by using 'Change Consensus'. Typically, the quality of these bases in the read and in the consensus is 99. That is so that next time phrap runs, it will correctly extend the consensus. However, if you aren't going to reassemble, you might want to just leave the quality values the way phred originally called them. You can do this by using a Consed parameter (consed.extendConsensusWithHighQuality), which you will learn more about later (see CONSED CUSTOMIZATION). 43) To delete a base, overstrike it with a '*' character. (Phrap ignores '*', so this is the same as deleting the character.) If you overstrike all bases in a column with * characters so the entire column consists of *'s (including the consensus base), there is no way to remove the column. This is OK since when you export the consensus (try the exercise on EXPORTING THE CONSENSUS), the *'s are not exported. While you are editing in Consed, we believe there should be a visual indication that a base was deleted. SAVING THE ASSEMBLY 44) To save the assembly, pull down the 'File' menu on the Aligned Reads Window, and release on 'Save assembly'. A box will pop up with a suggested name. I suggest you always use the one it suggests. The idea is that the ace files: (project).fasta.screen.ace.1 (project).fasta.screen.ace.2 (project).fasta.screen.ace.3 (project).fasta.screen.ace.4 (project).fasta.screen.ace.5 are in order of how old they are. If you feel you are taking up too much disk space, then start deleting the ace files starting at the oldest. I do not recommend that you overwrite existing ace files. The version numbers just keep growing, and that is not a problem. EXPORTING THE CONSENSUS 45) Exporting the consensus. Bring the Aligned Reads Window into view again. Hold down the left mouse button on the 'File' menu and release the button on 'Export consensus sequence'. Notice that the consensus will be stored (in this case) in a file called 'Contig1.fasta'. Click 'OK'. There is now a file in your edit_dir directory called 'Contig1.fasta' that has the consensus sequence in it. If you want to see the file, bring up another Xterm (if you are UNIX literate), and type: cd standard/edit_dir more Contig1.fasta 46) Fancier exporting the consensus. Bring the Aligned Reads Window into view again. Hold down the left mouse button on the 'File' menu but this time release on 'Export consensus sequence (with options)...'. Just export a little snip of the consensus, from 400 to 410. (You will notice this contains a pad * character.) Under "Write Both Bases File and Qual File or Just Bases File?" click "Both Files" Click 'OK'. Consed will want to call this file 'Contig1.fasta' again. You can overwrite the existing file. Look in your other Xterm at these files: more Contig1.fasta more Contig1.fasta.qual The one file contains the bases (but no * pads) and the other contains the corresponding qualities of those bases. 47) Exporting the consensus of all contigs at once: Go to the Main Consed Window. Point to 'File', hold down the left mouse button, and release on 'Write all contigs to fasta file'. You then can choose a filename for all contigs to be written to. (In this project there is only 1 contig, so there is no difference between this option and just exporting a contig at a time.) 48) COMPLEMENTING THE CONTIG Push 'Compl Cont' in the Aligned Reads Window to complement the contig. This displays the opposite strand of the contig including the consensus and all reads. Push this button again to uncomplement it. 49) COLOR MEANS EDITED AND TAGS (For this step, first click on the 'Dim' menu and release on 'Dim Nothing'.) Point to the 'Color' menu, hold down the left mouse button and release on 'Color Means Edited and Tags'. Notice that the bases that you have edited (make sure you have edited some bases) will stand out in either white or grey (depending on whether the base was made high quality or low quality). Observe this both in the Trace Window and the Aligned Reads window. This colormode is useful if you are interested in easily spotting which bases are edited. Return to the 'Color Means Quality and Tags' colormode by the following: point to the 'Color' menu, hold down the left mouse button and release on 'Color Means Quality and Tags'. FIND MAIN WINDOW 50) On the Aligned Reads window, click on 'Find Main Win'. This will cause the Consed Main Window to pop up in the event you have buried it under other windows or iconified it. (This may not work with some settings of your X emulator. In that case you will have to find and click on the Main Window to bring it up.) MULTIPLE UNDO EDIT 51) Now that the Consed Main Window is visible, click the 'Undo Edit...' button. There will be a popup indicating the most recent edit. (If it says "no edits so far", then bring up a trace and make several edits. Then click on 'Undo Edit...' again.) Click 'undo'. Then you will see the edit that was done before that. Click 'undo'. You can continue undoing if you like. You now know how to undo more than one edit. You cannot choose which edits to undo and which to not undo--edits can only be undone in precisely reverse order from the order you made them. Once you save the assembly, you cannot undo prior edits. SCROLLING TRACES AND ALIGNED READS TOGETHER 52) In the Aligned Reads window, scroll along the contig to a different point. Click the left mouse button on a read whose trace is already up. Notice that the existing trace instantly scrolls to the corresponding location. Now go to the Trace Window and scroll the traces to a new location. Click on the edt line with the left mouse button. You will notice that the Aligned Reads window will instantly scroll to the corresponding location. Thus you can keep the Aligned Reads window and the traces scrolled to the same location. SHOW ALL TRACES 53) Go to a region where there are lots of reads, say base 1660. Push down the right mouse button and release on 'Display traces for all reads'. You will see all traces displayed in a scrolling window. You can drag the scrollbar on the right down and up to see all the traces. This feature is particularly useful for polymorphism/mutation detection work. This feature was added to work in cooperation with polyphred. (See CONSED-POLYPHRED intereaction below.) In this Traces Window, point at one of the bases of one of the reads and click with the left mouse button. The base should start blinking in red. Now push the down arrow key on your keyboard. The cursor should move to the next read. Repeatedly type the down arrow key. Eventually the display should scroll so you can continue to see the read the cursor is on. Try the up arrow key as well. If there are more than 100 traces at a position, you will see those traces in batches of 100 traces. You can use the bottons at the bottom of the Traces Window labelled "prev 100 traces" and "next 100 traces" to move to the previous and next batches of 100 traces. There is also a button at the top of the Traces Window that changes between "Show All Traves" and "Show Just Good Traces". A "good trace" means a trace that is all of the following: * it has a base at the cursor location * the trace signal is sufficiently good * there is no trag on the read such as a dataNeeded tag that is listed in the resource: consed.showAllTracesDoNotShowTraceIfTheseTagsPresent: EXITING CONSED 54) On the Aligned Reads Window, point to 'File' menu, hold down the left button and release on 'Quit Consed'. If it asks you some questions, answer 'Quit Without Saving and Discard .wrk File'. ASSEMBLY VIEW 55) Consed can show you a bird's eye view of the Assembly using forward/reverse pair information, sequence match information, read depth, etc. We have a test database which shows its features. Type: cd assembly_view/edit_dir (You might need to type "cd ../.." first depending on where you are.) ls Restart consed Double click on "assembly_view.fasta.screen.ace.1" In the Consed Main Window, click on the button "Assembly View" which is near the upper left corner of the window. You should see 3 grey bars with pink labels "2", "3", and "1". The bars are the contigs: Pink "1" means Contig1, pink "2" means Contig2, etc. Notice the scale on the contigs. This gives the contig position. READ DEPTH 56) You should see two graphs above the contig bars: one bright green and one dark green. The dark green graph indicates read depth--the depth of the quality 20 (by default) region of reads. Turn off read depth as follows: Click on the button labelled "What to Show". A menu will popup at that location. Click on the "Read Depth" menu item. A box will appear labelled "Show Read Depth". It has a square (a toggle button) with "show read depth" to the right of the toggle button. Click on the toggle button to change it from appearing pushed in to appearing sticking out. Then click on "Apply". The read depth graph should disappear. If you would like, you can try showing read depth for other qualities other than 20. Note: the read depth is *not* the # of reads that have quality 20 bases or above, although this number is a good approximation. For example, suppose there is a stretch of 300 Q50 bases, and in the middle of that stretch are 5 Q10 bases. Those Q10 bases will be counted toward the Q20 read depth. (In computer science terms, these bases are part of the maximal Q20 read segment.) FORWARD/REVERSE PAIR DEPTH A "forward/reverse pair" is a pair of reads from the same subclone template, each of which is primed within the subclone vector, but one is primed on one side of the insert and the other is primed on the other end of the insert. A forward/reverse pair may both be assembled into the same contig, in which case they should point towards each other and be approximately the insert size apart. A forward reverse pair also might be in different contigs on different sides of a gap. 57) The bright green graph is highest around 7000 to 10000 of Contig2 and around 14000 of Contig3. The bright green graph indicates, for each base, the depth of subclone templates that have a consistent forward/reverse pair. A forward/reverse pair is "consistent" if the forward and reverse are pointing towards each other and are not too far away from each other. ("Too far" is defined as 3 or more standard deviations from the mean of the insert size of templates from a particular library.) In other words, the green graph tells for each base, how many consistent forward/reverse pairs have that base between the forward read and the reverse read. This forward/reverse pair depth is not the same as read depth, which is typically much less. Forward/reverse pair depth is important in that it gives a measure of the confidence of the assembly at a base. If the forward/reverse pair depth is close to zero, as it is in Contig1 position about 9300, there is a likelihood that phrap has made an incorrect join. When the forward/reverse pair depth is zero, the green line turns red, as it does on the right end of Contig3. INCONSISTENT FORWARD/REVERSE PAIRS 58) The red lines connect the right end of Contig3 with the middle of Contig1. These are filtered inconsisent forward/reverse pairs--they are "inconsistent" because they are not consistent (see above) and they are "filtered" in that they have another inconsistent read close by (at both ends) that is inconsistent for the same reason. If two red lines are on top of one another, it is displayed in purple so you know there is more than one there. This is a good example of a misassembly. There are many many reads at the right end of Contig3 that are paired with reads in the middle of Contig1. Notice that the forward/reverse pair depth of Contig1 is close to zero around base 9300. (You can use the "Zoom In" button to see this in more detail, but when you are done experimenting with the Zoom buttons and the scroll bar, click on "Zoom Orig" for the rest of this exercise.) This is where phrap made a bad join. If you tear the contig apart there, complement the left part of Contig1, and then join it to the right end of Contig3, the forward/reverse pairs will change from inconsistent to consistent. You will learn later how to do that. 59) Point to one of the red lines. You will notice that it turns yellow. the box near the bottom of the screen tells you a little more about what you have "highlighted" (turned yellow). If you want more information, click with the left mouse button. A window "Clicked Forward/Reverse Pairs" will appear giving information about each highlighted read. Try this. In the "Clicked Forward/Reverse Pairs" Window double click on one of the reads. The Aligned Reads Window should appear with the cursor on that read. This shows how to go from the Assembly View Window to the Aligned Reads Window. 60) You can also go from the Aligned Reads Window to the Assembly View Window. First you must make sure the Assembly View Window is already open (or else open it by clicking on Assembly View in the Consed Main Window). In the Aligned Reads Window, point to a read name, hold down the right mouse button, and release on "Find Read in Assembly View" (one of the last items in the menu the appears when you push down with the right mouse button). If the read is from a subclone that has a forward/reverse pair in the assembly, then the same "Clicked Forward/Reverse Pairs" Window will appear. It will contain not only the read that you pointed to, but all of the other reads from the same subclone as the one you pointed to. In the Assembly View Window, all of these reads will blink yellow. You can use this procedure to go within the Aligned Reads Window from forward read to reverse read or visa versa. 61) Notice the aqua and purple lines that connect the right end of Contig2 to the left end of Contig3. These are consistent gap-spanning forward/reverse pairs. If there is more than one pair on top of each other, the color is purple. These are the reads that tell you (and Consed, Autofinish, and Phrap) that the right end of Contig2 is connected to the left end of Contig3. As above, point to one to highlight it and click on it to see more information. 62) You can see much more information by clicking on the "What to Show" button, and then when the menu pops up, click on the "Fwd/Rev Pairs" menu item. Up will pop the "Which Fwd/Rev Pairs to Show in Assembly View" Window. Click on "All" next to "Show Inconsistent Forward/Reverse Pairs". Then click "Apply" at the bottom of this window. In this particular example, you just see a few more stray red lines. In a real example, you would probably see so many red lines that it would be a mess. In most cases those inconsistent forward/reverse pairs would be just caused by some laboratory problem (turning a plate around, mislabelling, etc) and not to any misassembly. Thus I suggest that you only generally leave "Show Inconsistent Forward/Reverse Pairs" to "Filtered". 63) Still in the "Which Fwd/Rev Pairs to Show in Assembly View" Window, click on "Show each consistent fwd/rev pair within contigs" (so the button looks as though it is pushed in) and click "Apply". This will show a blue (or purple if there is more than one at a location) square for each consistent forward/reverse pair within a contig. The horizontal position of the square is the center of the subclone (midway between the forward and reverse read) and the vertical position of the square indicates the size of the subclone (higher means a larger subclone). If you really want to see the position of the forward and reverse reads, you can do that too: Click on "Show legs on squares for consistent fwd/rev pairs" ("Show each consistent fwd/rev pair within contigs" must be still on) and click "Apply". What a mess! I believe most of this information is much more easily understood by just showing the "consistent fwd/rev pair depth" (the bright green graph described above). But it is your choice. When you want to highlight a consistent fwd/rev pair, you must point to the square--not the legs. Try it so you understand. 64) Suppose you have an assembly and there are some forward/reverse pairs that you specifically do not want to see in the Assembly View Window. For example, perhaps they are from a plate that was misnamed (or turned around) or from a library that is somehow less reliable. By hiding these forward/reverse pairs, the more reliable/important ones can more easily be seen. This is how you can do that: In the "Which Fwd/Rev Pairs to Show in Assembly View" Window, notice the line that says: Do not show templates in file doNotShowInAssemblyView.fof Underneath this are 3 buttons and probably the one that is selected is "show all templates". Try clicking "do not show specified templates" and click 'Apply'. See if you notice that anything changed in which forward/reverse pairs are displayed. If not, switch back and forth between "show all templates" and "do not show specified templates", each time clicking 'Apply'. When you see a line that appears and disappears, click on it to find what template it is. For example, djs736a2_fp04q146 is one such template. Then from an xterm in the assembly_view/edit_dir directory, type: more doNotShowInAssemblyView.fof You will see the names of the templates that are displayed/hidden. In order to hide particular forward/reverse pairs, put them into this file. This file can also contain the character '*' which means "match any characters". For example, djs736a1_fp* would match the template djs736a1_fp04q206 but not djs736a2_fp01q127 65) Try turning on/off each of the Fwd/Rev Pair options so you understand them. (In this example, there are no "consistent fwd/rev pairs between different scaffolds.") SEQUENCE MATCHES 66) Notice the curvy orange lines connecting Contig1 with Contig2 and Contig3. These show sequence matches. Point at the one connecting Contig1 and Contig2 and click on it. A "Sequence Matches" box will popup saying that this match has 119 bases and has a similarity of 90.8%. Click on that line so its background turns black. Then click on the button "Show Alignment". Up will pop the Compare Contigs Window with the alignment shown in the lower half of this box. You will learn more about this later (see "JOIN CONTIGS"). For now, dismiss this window. 67) In the Assembly View Window, click on "What to Show" and then when the menu pops up, click on "Sequence Matches". In the "Which Sequence Matches to Show in Assembly View" Window, try clicking off "ok to show sequence matches between contigs". Then click the "Apply" button. You should see the orange lines disappear. (Any highlighted lines will not disappear.) Click "ok to show sequence matches between contigs" back on, and click "Apply" and the lines should be back. 68) Also in the "Which Sequence Matches to Show in Assembly View" Window, change the minimum similarity from 90 to 85. Click "Apply". You should see a lot more orange curvy lines, and now you should also see black curvy lines. If you look carefully, you will see that 2 lines within each pair of orange curvy lines do not cross each other but the 2 lines within each pair of black curvy lines do. This is because orange is used to show direct repeats and black is used to show inverted repeats (relative to the orientation of the contigs in the Assembly View Window). 69) Also in the "Which Sequence Matches to Show in Assembly View" Window, click on "filter seq matches by size" and set the min size to 400 and the max size to some huge number (e.g., 1000000) and click "Apply". You will see just one direct repeat (orange curvy lines) of size 745. 70) Try some of the other ways of filtering the sequence matches on "Which Sequence Matches to Show in Assembly View". 71) You must learn this step if you are going to ever see sequence matches with your own data, so don't skip this step. If you have problems, it is likely that the phred/phrap/consed package has not been installed correctly and you will need help from your system administrator. Exit Consed and look at the files in assembly_view/edit_dir. Notice there is a file: assembly_view.fasta.screen.ace.1.aview This is what Consed uses to show sequence matches in the Assembly View Window. When you use your own data, you will not have this file so you will need to learn how to create it. Hide it from Consed by (in practice you will never do this step--this is just to simulate the .aview file not being there): mv assembly_view.fasta.screen.ace.1.aview assembly_view.fasta.screen.ace.1.aview_hide Now restart consed and select ace file assembly_view.fasta.screen.ace.1 If you are asked if you want to apply edits, click the "No" button. Click on "Assembly View" in the Consed Main Window. You will get the error message: "Sequence matches will not be shown in Assembly View because there is no file assembly_view.fasta.screen.ace.1.aview If you want sequence matches to be shown, click on "What to show: Sequence Matches" and then "run crossmatch" 72) RUNNING CROSSMATCH FOR SEQUENCE MATCHES Just as the instructions (above) say, click on "What to show" and then when the popup menu appears, click on "Sequence Matches" and then when the "Which Sequence Matches to Show In Assembly View" Window comes up, click on the "Run Crossmatch" button. Watch the action in the xterm. There should be several pages worth of output from crossmatch that scrolls by in the xterm. If you get an error, it is likely that the phred/phrap/consed package is not correctly installed. You (or your system administrator) should track down the problems and correct them. If you are successful, then 3 orange pairs of curvy lines will appear in the Assembly View Window--the same as you saw in the steps above. PULLING OUT READS AND RE-ASSEMBLYING THEM (MINIASSEMBLIES) When the Assembly View Window indicates, using forward-reverse pair information, that there is a misassembly, Consed provides the tools to correct that misassembly: you can first pull out the the misassembled reads from their current contigs into individual contigs, with a single read per contig. Then you can reassemble those new contigs that each contain a single read. Let's do this: 73) In the Assembly View Window move your cursor so that the red and purple forward/reverse pair lines turn yellow. You will be unable to get them all yellow, but get as many as you can. Then click with the left mouse button. A window labelled "Clicked Fwd/Rev Pairs" should appear with a very long list of reads in it (around 53 reads). 74) In the "Clicked Fwd/Rev Pairs" Window, click on the button labelled "Pull out reads". A window labelled "Put Reads into Their Own Contigs" should appear. 75) In the "Put Reads into Their Own Contigs" Window, select all of the reads. You can do that by clicking with the left mouse button on the first read and then scrolling down to the bottom of the list of reads, holding down the shift key and clicking with the left mouse button on the last read. (When a read is selected, its background should be black.) Click on the button "Remove Highlighted Reads". The Assembly View Window will close and reopen after a few seconds and will complain about not being able to show sequence matches. Save the assembly (see "SAVING THE ASSEMBLY" above) and follow the instructions in "RUNNING CROSSMATCH FOR SEQUENCE MATCHES" (above). The assembly will now probably contain 4 contigs: 2-3-1c in one scaffold and 4 in the other. That is because when the misassembled reads were pulled out of Contig1, it fell into two new contigs: the new contig 1 and contig 4. All of the reads you pulled out have created Contig5, Contig6, ... and approximately Contig58, each of which contain only a single read. MINIASSEMBLIES 76) On the Consed Main Window, click the button "Miniassembly". A box will popup labelled "Reassemble Some Contigs". On the left part of the box will be all contigs, from Contig1 to about Contig58. Notice that starting with Contig5 will be contigs that contain only a single read. On the right will be Contig5 through approximately Contig58. You add or delete from the list on the right. For example, to delete Contig5 from the list on the right, click on it, and then click "Clear Highlighted". The right list should now only contain Contig6 through the last contig. Add Contig5 back to the right list by clicking on Contig5 in the left list and then clicking on the button labelled "Move Highlighted to Right". Contig5 will now appear at the bottom of the list on the right. 77) Leave all of these boxes blank: "-minscore", "-minmatch", "-forcelevel", and "other phrap options:". Keep "Put into separate contigs" selected rather than "Disgard from assembly". Click the "Reassemble" button. If you haven't saved the assembly, a box will popup saying "Error You must first save the assembly before making a miniassembly". Follow the instructions you learned above ("SAVING THE ASSEMBLY") to save the assembly. Then click the "Reassemble" button again and watch the action in the xterm. Lots of output from determineReadTypes.perl, phrap, crossmatch will scroll by in the xterm as those programs run. (If they don't, you haven't correctly installed all of the Consed package.) 78) When the miniassembly is complete, a box will popup asking "Would you prefer to discard this miniassembly and reassemble again?" Click the "No" button. 79) On the Consed Main Window, click the "Assembly View" button. Consed will complain about not being able to show Sequence Matches so save the assembly and follow the instructions in "RUNNING CROSSMATCH FOR SEQUENCE MATCHES" (above). In the Assembly View Window in addition to Contig1, Contig2, Contig3, and Contig4, you should see a few more contigs. These are the result of the miniassembly of all those individual reads. CONTIG ARRANGEMENT--REORDER CONTIGS Contigs are arranged by Consed into "scaffolds" using forward/reverse pair information. However, you might have some external information (such as digest information) that tells you a different arrangement. You can use Consed to rearrange the contigs. This new arrangement will be preserved even if you reassemble. 80) Exit Consed and then restart Consed. Double click on "assembly_view.fasta.screen.ace.1" (If a window pops up saying "There is an edit history file ( a .wrk file )...", click the "No" button.) Click on the "Assembly View" button. You will see two scaffolds: one on the top row with Contig2 and Contig3, and one on the bottom row with just Contig1. Now suppose that you believe that Contig2 and Contig1 are connected together instead of Contig2 and Contig3. To do this: 81) Within the Assembly View Window, click on the "Contig Arrangement" button. Up will pop a menu. Click on "Reorder Contigs". A "Reorder Contigs" Window will pop up. Enter the following information: Contig: 2 [Right End] connected to Contig: 1 [Left End] That is, you must enter "2" and "1" in the contig boxes, and you must click on the first "right end" button. Then click on the "Add and Restart Assembly View" button. A warning box will pop up telling you that you are crazy, because there are 12 forward/reverse pairs as evidence that the scaffold as displayed in the Assembly View Window is already correct. Click on "yes"--that you are sure. The Assembly View Window will disappear for a second and reappear, with Consed2 and Contig1 connected together, just as you wanted. CONTIG ORIENTATION 82) Some users want a scaffold oriented a particular way. For example, one user might be working on a particular gene so wants to always view the top strand of that gene. Another user might be finishing a BAC and wants the 5' end of the BAC on the left of the scaffold. Phrap, however, may not respect their wishes and might have contigs complemented from the way the users want to view them. Consed provides a way for the user to indicate his/her desired orientation, and thereafter if phrap complements a contig from that desired orientation, Consed will complement the contig back when Consed starts up. To demonstrate this, exit Consed and then restart Consed. Double click on "assembly_view.fasta.screen.ace.1" In the Consed Main Window, double click on Contig1. You will see read djs736a2_fp02q494.y1 pointing left. But let's suppose that you would rather the Contig be in the other orientation, with read djs736a2_fp02q494.y1 pointing right. In the Consed Main Window, click on Assembly View. Then click on the button labelled "contig arrangement". When a popup menu comes up, click on "Reorient Contigs". The "Reorient Contigs Window" should come up. Highlight the scaffold labelled "1" under "Select a scaffold". Click on "flip scaffold". Then push the button labelled "Apply and Restart Assembly View". There will be an error box complaining about not being able to show sequence matches. To fix that, save the assembly and follow the instructions in "RUNNING CROSSMATCH FOR SEQUENCE MATCHES" (above). In the Consed Main Window, double click on Contig1 so the Aligned Reads Window comes up. Scroll to the right end. You will notice that djs736a2_fp02q494.y1 is now on the right end pointing right. What is the difference between doing this and just complementing the contig, which just requires the click of a button? The difference is that complementing the contig will be undone the next time phrap runs, but using this procedure will be permanent, even if phrap complements the contig. RESTRICTION FRAGMENTS We'll look at this feature in Assembly View after we've learned how to use the Restriction Fragment Window. CONSED-POLYPHRED INTERACTION Polyphred is a program for finding polymorphic sites; it was developed by Debbie Nickerson's group (contact them at http://droog.mbt.washington.edu). We have a test database, 'polyphred', which has had polyphred run on it already. Polyphred has put a polymorphism tag on each polymorphic site. If Consed is running, exit it. Type: cd polyphred/edit_dir (You might need to first type "cd ../.." depending on where you are.) ls Restart Consed. Double click on example2.fasta.screen.ace.1 When Consed comes up, you should see 2 contigs. Double click on Contig2 In the Aligned Reads Window, push the left mouse button while pointing to the 'Navigate' menu and release on: 'Toggle feature: when navigating to consensus location, pop up all traces (currently off)' That will turn this feature on. Now push the left mouse button while pointing to the 'Navigate' menu and release on 'Tags'. Up should pop a list of tag types. Double click on 'polymorphism'. Polyphred has already been run so the consensus is tagged with polymorphism tags at each polymorphic site. Up will pop a window labelled 'Polymorphism Tags' with a list of sites. Click on 'Next'. If you correctly followed the instructions above, all the traces should pop up at the first polymorphic site. You may want to reposition the traces window to see it better. Now ignore the original 'Polymorphism Tags' window and instead click on 'Next' in the *traces* window. This will take you to the next polymorphic site. Pretty nice, huh? Dismiss the Traces Window. 83) ALPHABETICAL ORDERING OF READS The reads can be ordered in 3 ways: a) alphabetically b) first all the top strand reads and then all the bottom strand reads. The top strand reads are then ordered by the left end of the reads. Same with the bottom strand reads. c) arbitrarily by a user-provided file Try changing between a) and b). In the Consed Main Window (click on 'Find Main Win' on the Aligned Reads Window if you can't find the Main Consed Window because it is covered up with other windows), pull down the 'Options' menu, and release on 'General Preferences'. Scroll down until you find 'Display reads sorted alphabetically or by strand/left end of read.' Switch it between 'alpha' and 'strand'. Then click 'Apply and Dismiss'. Notice the effect in the Aligned Reads Window. Many polymorphism and mutation detection labs find that alphabetically sorting is most useful, while many genomic sequencing labs find that sorting by strand/left end of read is most useful. If you want to use a user-provided file, you must learn CONSED CUSTOMIZATION (below) with resources: consed.showReadsInAlignedReadsWindowOrderedByFile: false consed.showReadsInAlignedReadsWindowOrderedByThisFile: readOrder.txt After you are done playing with these features, exit Consed and go back to the previous database: cd standard/edit_dir (You might need to first type "cd ../.." depending on where you are.) ls Restart Consed. Double click on standard.fasta.screen.ace.1 When it says "There is an edit history file (a .wrk file)...Do you want to apply those edits?", click on "no". Double click on Contig1 to bring up the Aligned Reads Window again in preparation for the next step. NAVIGATING 84) In the Aligned Reads window, pull down the Navigate menu and release on 'Low consensus quality'. You will see a list of locations. Move the 'Low consensus quality' window down so you can see the Aligned Reads window. Repeatedly click on 'Next' until you reach the end of the list. (Low consensus quality means an area in which the bases each have too high probability of being wrong.) This saves you from having to look through large amounts of high quality data trying to find problem areas. There are 2 'Next' buttons--one on the Aligned Reads Window and one on the Low Consensus Quality Window. You can click on either, but it is probably more convenient to use the 'Next' button on the Aligned Reads Window. Thus you can keep the Aligned Reads Window in front with input focus and keep the Low consensus quality window pushed out of the way. You may want to click on the 'Save' button in the Low consensus quality Window to save to a file a copy of this list of problem areas as you work through them. In our experience, this will be the most important navigate list you will use. In fact, finishing partly consists mainly of adding reads and rephrapping until this list is reduced to nothing. 85) Dismiss the Low consensus quality window. Pull down the 'Navigate' menu again and release on 'High quality discrepancies as above, but omitting tagged compressions and G_dropouts'. You will probably notice there are no entries (unless you created some yourself by editing). That is because there are no high quality discrepancies with this dataset. So let's force there to be some by lowering the quality threshold. First, dismiss the High quality discrepancies window. Click on 'Find Main Win'. In the Consed Main Window, pulldown the 'Options' menu and release on 'General Preferences'. Notice that the default for 'Threshold for High Quality Discrepancy' is 40. Change it to 15 and click 'Apply & Dismiss'. Then follow the steps above to bring up the High quality discrepancies menu. Now you will see several entries. Click 'next' repeatedly to go successively to the next high quality discrepancy in the Aligned Reads Window. You can also double click on a particular line in the High quality discrepancies window to go to that location. Alternatively, you can single click on a line and then click the 'Go' button. Dismiss the High quality discrepancies window. 86) Similarly, try the other navigate lists: Unaligned high quality regions (this list will be empty with this data set), Edits, Regions covered by only 1 strand and only 1 chemistry, and Regions covered by only 1 subclone. Unaligned high quality regions are regions in which the traces are high quality so there is no question of the bases, but the region differs so much from other reads that phrap has given up trying to align the region with the consensus. This could be due to a chimeric read, or perhaps the read belongs somewhere else. We believe that regions covered by only 1 subclone should be covered by a 2nd subclone to prevent the possibility of there being a deletion in the single subclone. There are so many different problem lists that you may forget to check one of them and thus miss a serious problem. Thus we combined them all into a single list. This is the first menu item: 'Low Cons/High Qual Discrep/Single Stranded/Single Subclone/Unaligned High'. We suggest you use this list. 87) Also try navigate by tags by selecting 'tags' under navigate: when the Select Tag Type Window appears, double click on 'compression'. (Note that you can't do anything else until you deal with this window.) This gives a list of a particular tag type in a particular contig. 88) There is also a way of getting a list of a particular tag type in all contigs: Click on 'Find Main Win'. In the Consed Main Window, point to the 'Navigate' menu, hold down the left mouse button, and release on 'Tags in all contigs'. Continue as in the previous step. (Since there is only one contig, this list will not be any different than the corresponding list for Contig1.) PRIMER-PICKING 89) Go to some location near the right end of the contig, say base 2470. Click with the right mouse button on the consensus and click on either one of the top strand primer choices (either from subclone template or from clone template). Consed will pause a moment, and then there will appear a selection of primers that pass all of Consed's requirements. (If you get an error message, Consed might not have been correctly installed. See INSTALLING CONSED above.) Templates are also chosen for each primer. You may have to scroll the primer list to the right to see the templates. Consed lists these templates in order of quality--all of them will cover the read you want to make. Double click on one of the primers in the Primers Window. That will cause the Aligned Reads Window to scroll to show that oligo in context. Click on 'Accept Primer'. A comment box will pop up. Enter some comment and click 'OK'. Notice that a yellow oligo tag, with a little red end, is created on the consensus for that primer. The red end points in the direction of the oligo. The tag contains all the information you need to order that oligo and do the reaction--you will learn how to pop it up below under 'tags'. What is the difference between 'Pick Primer from Subclone Template' and 'Pick Primer from Clone Template'? There are 3 differences: A. which vector file the primers are screened against. In the former case, the primer is screened against the file primerSubcloneScreen.seq and in the latter case against the file primerCloneScreen.seq B. In checking for false matches elsewhere in the assembly, if the template is the whole clone, then Consed must check for false matches in the *entire* assembly, including all other contigs. But if the template is just going to be a subclone, Consed only needs to check elsewhere in that subclone. Actually, to be conservative, Consed checks for false matches +/- the maximum insert size of a subclone. C. If you are picking primers for subclone template, then the primer picker can also pick the subclone templates. If it doesn't find any suitable subclone template, it will reject the primer. (By default, picking of subclone templates is turned on. If you prefer to pick your own templates, and want Consed's primer picker to be much faster, you can turn it off temporarily or permanently. To turn it off temporarily, go to the Consed Main Window, point to the Options menu, hold down the left mouse button and release on 'Primer Picking Preferences'. Scroll down to 'Pick Subclone Templates for Primers' and click 'False'. Click on 'Apply and Dismiss'. To change this permanently, see CONSED CUSTOMIZATION below. Beware: you must correctly customize determineReadTypes.perl for template picking to work. See INSTALLING CONSED above.) If you are interested in the details of primer-picking, see the section 'AUTOFINISH AND PRIMER PARAMETERS' (below). When you are done editing and have saved the assembly and exited Consed, run ace2Oligos.perl (supplied with this distribution--make sure your system administrator installed it) which will extract all the oligos you just created. This is handy for email ordering of oligos. In the xterm, type: ace2Oligos.perl standard.fasta.screen.ace.2 oligos.txt where standard.fasta.screen.ace.2 is whatever the name is of the ace file you just saved. ace2Oligos.perl does not record the comments that the finisher entered when creating the oligo. If you want to record that as well, you could use the script ace2OligosWithComments.perl which was written by a Consed user and thus is found in the "contributions" directory. 90) WHEN CONSED CAN'T FIND AN ACCEPTABLE PRIMER Sometimes Consed refuses to pick a primer. This is because it has tried every possible primer and rejected it for one reason or another. If you don't understand why it didn't pick a particular primer, you can ask it as follows: In the Aligned Reads Window, point to the "Misc" menu, hold down the left mouse button and release on "Check Primer". Enter the left and right consensus positions of the primer, check which strand, and whether the primer is to use subclone templates or the whole clone as a template. Consed will tell you all that is wrong with that primer. Try looking at a top strand subclone primer from 2340 to 2360. 91) PICKING PCR PRIMER PAIRS In the Aligned Reads Window, go to the location where you want to pick the first PCR primer, say base 500. Point to the consensus, hold down the right mouse button and release on "Top Strand PCR Primer". Then scroll to the location where you want to pick the second PCR primer, say base 2200. Point to the consensus, hold down the right mouse button and release on "Bottom Strand PCR Primer". There will be a pause and then there will be a list of PCR primer pairs. Click on the pair you want and click "Accept Pair". You can modify the parameters for choosing PCR primer pairs by going to the Consed Main Window, pointing to "Options", holding down the left mouse button, and releasing on "Primer Picking Preferences." For example, by default Consed does not display all PCR primer pairs--this would take too long and give you too many. However, you can ask it to show you all such pairs. In the Primer Picking Preferences, scroll down to "Check All PCR Pairs (huge) or Just Sample?" and click on "All". Then click on "Apply and Dismiss". Then pick PCR primers again, as above. Don't be surprised if you get 10,000 or more pairs of primers! (PCR Primers are screened for: melting temperature and length, the melting temperature of the 2 primers must be sufficiently close to each other, each primers must not stick to itself or to the other primer, no mononucleotide repeats, only ACGT's (no n's or ambiguity codes), and primer pair must not amplify any other location. There are many more details...) SEARCH FOR STRING 92) Try the 'Search for String' button (left side of the Aligned Reads Window). Type in a string (such as aaaca), and click 'ok'. There should be a list of 'hits'. Double click on one of the hits (or single click on it and click on 'go'.) Notice that the Aligned Reads Window scrolls to that position and has the cursor on the found string. (It might be complemented.) Dismiss this window. Try this again, only this time in the Search For String Window select 'Search Just Reads'. Then click 'OK'. You will notice there are many more hits. This is because this shows hits in each read, even if they are at the same consensus position. You can also try the approximate match search for string by clicking on 'Approximate' instead of 'Exact'. The 'Per Cent Mismatch' only applies to the Approximate match search. COPY AND PASTE 93) In the Aligned Reads Window, swipe some bases by holding down the left mouse button. You should see the bases turn yellow, at least temporarily. Then click the 'Search for String' button. Use the middle mouse button to paste the bases you have just swiped into the 'Query string:' box. Notice that you can swipe bases either from the consensus or from a read. The search for string is case-insensitive so don't worry about the pasting being upper or lowercase. CORRECTING FALSE JOINS MADE BY PHRAP 94) Phrap may put several reads together that you believe do not belong together. (For example, you may see several high quality discrepancies between the reads.) If you are sure these reads do not belong together, you can force a subsequent reassembly by phrap to not assemble those reads together. You do this by finding a location where there is a high quality discrepancy. Then click on the read with the right mouse button and release on 'Tell phrap not to overlap reads discrepant at this location'. There are no high quality discrepancies with this dataset so Consed won't let you do this. (Try it and see.) However, when you use your own data, you may get the chance! It is possible to automate this procedure using AutoEdit (see USING AUTOEDIT). ADD NEW READS 95) For this to work, your system administrator must have set up everything correctly. (See below in INSTALLING CONSED.) Assuming you have set everything up correctly, you can now experiment with adding reads. From a UNIX prompt, copy the new chromatograms into the chromat_dir directory: cp ../chromats_to_add/* ../chromat_dir Exit Consed and bring it up again using the original ace file standard.fasta.screen.ace.1 If it asks if you want to apply edits, just say 'no'. On the Main Window, click on the Add New Reads button. There will appear a list of files ending with .fof. These are files that contain lists of chromatograms. Double click on 'reads_to_add.fof' Then Consed will ask "If a read doesn't align against any existing contig, do you want to have it go into a contig by itself? (otherwise it will just not be put into the assembly)" Users usually prefer to answer "yes". Consed will ask "Do you want to recalculate the consensus quality values where each of the new reads is aligned?" Answer yes or no, but in practice you should generally answer "yes." There should be lots of progress output in the xterm from which you started Consed. When it completes, there will be a Reads Added Window popup with a report of which reads were added. In this case, it should say that 9 reads were successfully added and list them. If you get an error message, look carefully at the full error message in the xterm to diagnose the problem. Probably there is some mistake in how you installed Consed. See INSTALLING CONSED (above). TEAR CONTIG Just so you get the same results as I do, exit Consed and bring it up again using the original ace file standard.fasta.screen.ace.1 If it asks if you want to apply edits, just say 'no'. 96) When phrap really screws up, you may want to just tear the contig apart in several places and then join the pieces back together in a different way. Let's try it: Go to location 1500. Point the mouse at the consensus base at 1500 and push the right mouse button down. Release the button on 'Tear Contig at This Consensus Position'. Up will pop a list of reads with 2 little buttons next to them <- and ->. Leave everything as it is and just click 'Do Tear'. (If you want to play around with which reads goes into which contig, do that another time.) Now you should have 2 Aligned Reads Windows on top of each other. One should contain 'Contig2' and the other 'Contig3'. Dismiss the little window that says 'Tear Complete'. JOIN CONTIGS 97) Now let's join these 2 contigs back together: Click on 'Search for String' and type in the following bases: agctgccatc Click 'OK'. Search for string should find 2 locations, one in Contig2 and one in Contig3: Contig2 (consensus) 1447-1456 (uncomplemented) Contig3 (consensus) 829-838 (uncomplemented) Double click on the first one. The Aligned Reads Window for Contig2 will scroll to location 1447 and the window will raise up. In that Aligned Reads Window, click on 'Compare Cont'. Now double click on the 'Contig3' line in the above Search for String results. The Aligned Reads Window for Contig3 will scroll to location 829 and lift up. In that Aligned Reads Window, click on 'Compare Cont'. Now the Compare Contigs Window should be visible. In the Compare Contigs Window, try scrolling back and forth. You can change the cursors (blinking red), but if you do, please return them to the locations 1447 and 829 for the next step. The cursors 'pin' these bases together when doing an alignment. (The algorithm is a pinned and banded Smith-Waterman alignment.) Click on Align. Try scrolling the alignment by dragging the thumb in the lower half of the Compare Contigs. An 'X' means there is a discrepancy between the 2 contigs. There is also a 'P' (see if you can find it!) The P indicates the bases that you pinned together. You will also notice that some bases are lighter and some are darker. This indicates quality just as in the Aligned Reads Window. You will notice that wherever there an is a discrepancy (an 'X') one of the bases is low quality. This is your cue that the discrepancy is just a base calling error rather than indicating that the two contigs really are different but similar locations. Click with the left mouse button on either contig in the bottom alignment. You will notice that both contigs will have the red blinking cursor in the same position. Click on 'Scroll Both Aligned Reads Windows' and look at the Aligned Reads Windows to see that they scroll to the corresponding positions. You can have traces up for the contigs, and they will scroll as well. Experiment with this. Then click 'Join Contigs'. The 2 previous Aligned Reads Windows will disappear and there will be a new one which has a new contig 'Contig4'. You have made a join! Scroll left and right. You will notice that many of the reads are highlighted. These are the reads that came from the previous "right" contig. To unhighlight all of these reads at once, point to the "Misc" menu, hold down the left mouse button and release on "Unhighlight All Reads". It is possible to have more than one Compare Contigs Windows up at a time. This allows you to investigate a repeat that has more than 2 copies. COMPARE CONTIGS WINDOW AND INVERTED REPEATS In the above example, we used the Compare Contigs Window to examine a sequence match between two different contigs. It is also possible to use the Compare Contigs Window to examine a sequence match between two copies of a repeat within the same contig, either direct or inverted. 98) To see this, restart Consed: ../../consed_(computer type) Double click on standard.fasta.screen.ace.1 When it says "There is an edit history file (a .wrk file)...Do you want to apply those edits?", click on "no". Double click on Contig1 to bring up the Aligned Reads Window. Go to position 69 (use the "Pos:" box described above). Click the "Compare Cont" button on the Aligned Reads Window. The Compare Contigs Window will popup, but move it aside. Go to position 2035 in the Aligned Reads Window. Click the "Compare Contig" button again on the Aligned Reads Window. In the Compare Contigs Window there are two copies of Contig1--one on top and one on the bottom. Each has a "complement just in this window" button. Click on the bottom one (the one that has position 2035 blinking red). After clicking on it, you should notice that the numbers on the bottom contig are reversed to they decrease to the right--a copy of Contig1 has been reversed and complemented. Now click the "Align" button. Suddenly, you should see the alignment appear in the bottom half of the Compare Contigs Window. You should see bases between 69-78 aligned against the reversed complement of bases from 2026-2035. This has shown how you explore an inverted repeat. If you wanted to examine a direct repeat, you would use the same method except you wouldn't click on the "complement just in this window" button. Compare Contigs is one method of exploring joins of contigs that were not made by phrap. Another method is to use the Assembly View Window (above). They are designed to work together: the Assembly View Window gives a high level view of all sequence matches and takes you to the Compare Contigs Window which shows the alignment of a single sequence match and, if the user so desires, makes a join. REMOVING READS 99) You can remove individual reads and put them into their own contigs. For example, in the Aligned Reads Window, go to location 2000. Point to the read name of read djs74_2664.s1 and hold down the right mouse button. Release on 'Put read djs74_2664.s1 into its own contig.' Presto-chango! The read is put into its own contig and the old contig is redrawn without the read in it. At this point you should save the assembly--you should always save the assembly after removing reads. 100) You can also remove many reads at once. Look at the Consed Main Window. Click on "Remove Reads". Type into the "File of read names:" box "reads_to_remove.fof" and either push the "Enter" key or click on "Read File". You should see a list of 2 reads: djs74-2231.s1 djs74-3174.s1 You can click back and forth between the choices of "Delete Reads from Assembly" and "Just Put Each Read into Its Own Contig". Try each one. Delete Reads from Assembly means that the read will no longer appear in Consed. When you are using your own data and you really want to remove reads from the assembly, you must also use the UNIX "rm" command to remove the corresponding phd files from phd_dir and the chromatograms from chromat_dir. Otherwise, the next time you run phredPhrap, the reads, like Phoenix, will rise again to become part of the next assembly. After you have completed this exercise, restart Consed so that you have all the reads in their original locations for the following exercises. TAGS 101) Bring up a trace for a read (as above). Swipe some bases on the 'edt' line while holding the middle mouse button down. A list of choices will pop up. Select 'Add Tag'. Type in a comment in the box at the bottom, and select 'comment' from the list of tag types. You will now see a blue box both in the Aligned Reads Window and in the Traces Window on that read. To see the comment, you can just point to it in the Aligned Reads Window and you will see the comment in the lower right hand corner of the Aligned Reads Window. Alternatively, you can click on that blue tag in the Aligned Reads Window with the right mouse button and release on 'Tag: comment Show more info?'. Alternatively, you can click on the blue tag in the Traces Window with the right mouse button. Try creating some other kinds of tags: again swipe some bases in the Trace Window by selecting a different tag type. You will notice that different tags are in different colors. You can always use the methods above to see what kind of tag it is if you forget what a particular color means. You can also define your own tag types. See below CREATING CUSTOM TAG TYPES for how to do that. CREATING LONG TAGS 102) You can create really, really long tags as follows: Just create a short version of the tag as above for where you want the tag to start. Then figure out the consensus position of where you want the tag to end. In the Aligned Reads Window, click on the short tag with the right mouse button and release on 'tag: show more info?' (as above). A Tag Window will appear for that tag. In the Tag Window, simply change the End Unpadded Consensus Position to the place you want it to end. Then click 'OK'. You will now notice that the tag will be as long as you wanted. CONSENSUS TAGS 103) You can create tags on the consensus in the same way. In the Aligned Reads Window, use the middle mouse button to swipe some bases on the consensus in the Aligned Reads Window. Up will pop a list of tag types. Click on one of them. Try it again somewhere else. Try it with the tag type being 'comment'. In this case, you must enter a comment. Notice the pretty colors! If you forget which tag type a particular color represents, just point at the colored tag with the mouse and the tag type will be displayed at the bottom of the Aligned Reads Window. 104) Try creating some tags that overlap each other. You will notice that the overlapping region will be purple. If you want to know which tags overlap, you can use any of the methods already discussed. SEARCH FOR READ NAME 105) Restart Consed using the original ace file standard.fasta.screen.ace.1 If it asks if you want to apply edits, just say 'no'. Instead of clicking on a read or contig name, type a read name into the "Find reads containing (*'s allowed):". If you want to look at the location containing read djs74-2689.s1, you can just type "2689" and then push the "Enter" key and Consed will immediately bring up the Aligned Reads Window with the cursor on read djs74-2689.s1. Suppose that there were more than one read that matched? For example, suppose you type: "26" and then push the "Enter" key. This matches 3 reads: djs74-2689.s1 djs74-2679.s1 djs74-2664.s1 Try it and see what happens... Try entering "26*9" and see what happens. What does the "*" mean? Try using "Find 1st read starting with:". Try typing djs74-2 You will notice that as you type each letter, the first item in the list that matches the letters typed will be highlighted. Experiment with deleting a few letters and typing others. This is a powerful method of quickly getting to the read name you are interested in. When you get to the name in the list, you do not have to type the rest of the name--just type carriage return or else click on 'OK'. ONLINE DOCUMENTATION 106) On the Aligned Reads Window or on the Consed Main Window, click on the 'Help' menu and release on 'Show Documentation'. You will see this document. You can search for keywords in it. It is also on the web. Go to http://bozeman.mbt.washington.edu/consed/consed.html, and find "complete documentation" near the bottom of the page. THE .WRK FILE 107) Consed keeps a log of all changes you make to an assembly: adding new reads, putting reads into their own contigs, making joins and tears, adding and removing tags, and changing bases. This log is kept in a file ending with ".wrk". You can use this file to help you remember exactly what you did to an assembly. 108) You should save your edits by pulling open the 'File' menu on the Aligned Reads Window, and releasing on 'Save assembly'. RESTRICTION DIGEST 109) Restart Consed. Double click on "standard.fasta.screen.ace.1" In the Consed Main Window, click the "Digest" button. For the purpose of this exercise, the full pathname of file of vector sequence can refer to any file of sequence in fasta format. However, when you are using it with your own data it should refer to a file that contains the sequence of your cloning vector. For example, if you are sequencing a BAC, it should contain BAC vector. The sequence must start at the vector/insert junction that you used when you ligated the insert. Click "OK". You will see a comparison of in-silico fragments (those calculated from the sequence) and real fragments (those in fragSizes.txt which supposedly came from a real gel). * If a band is red, that means that it doesn't match. * If a band has a "v" on it, that means it is a vector fragment. * If a band has a "g" on it, that means it is a gap-spanning fragment. Move the pointer over the fragments, and you will see the fragment sizes appear. Move the pointer to the in-silico fragment with size 2299. Click on it. You will see the fragment on the left size of the window become highlighted. Click on the button labeled "right end" (2nd row from the bottom of the window) and the Aligned Reads Window will pop up, with the cursor on the right end of the fragment. Click on "show problems" and navigate through the list of problems by clicking on "next". You will notice that the Gel Window is zoomed in. To return to the original zoom, click on "Zoom Original". Where it says "Select Enzyme:", point to "EcoRV", hold down the left mouse button and release on "HindIII". This is how you change enzymes. Click on the button labeled "Text Output". This can be saved to a file and printed out. Dismiss the restriction digest window. On the Consed Main Window, click the "Digest" button again. Notice the file "fragSizes.txt". This is a file of actual gel fragment sizes. If you don't have an actual gel, but rather you want to just make predictions of fragment sizes from the sequence, you can leave this box blank (erase the "fragSizes.txt"). Try that. fragSizes.txt has the following format: >EcoRV 448 710 1102 1197 -1 >HindIII 448 508 586 735 801 -1 where EcoRV and HindIII are enzymes and the numbers below them are the actual fragment sizes. Each enzyme list is terminated by -1. Consed does its best to try to figure out which end of the clone insert is connected to which end of the vector. However, it sometimes is wrong. If you believe it is wrong, you can click "compl vector" to try connecting the insert to the vector in the opposite orientation and see if that produces better agreement with the actual digest. RESTRICTION DIGEST AND ASSEMBLY VIEW 110) Go to the assembly_view sample dataset and bring up the Assembly View Window: cd assembly_view/edit_dir (You might need to type "cd ../.." first depending on where you are.) ls Restart consed Double click on "assembly_view.fasta.screen.ace.1" In the Consed Main Window, click on the button "Assembly View" which is near the upper left corner of the window. Also on the Consed Main Window, click on Digest. The "Select Enzyme and Contigs" Window should appear with EcoRV and HindIII selected. Click OK. The "Display Digest" Window should appear. Now look at the Assembly View Window. You will notice blue, green, and red rectangles under the grey contig bars. These rectangles are the in-silico restriction fragments. Point to one of them-- it will turn yellow and information will be displayed in the information box below. Point to one of the EcoRV fragments, hold down the right mouse button, and release on "Goto fragment in digest window". Notice that in the Display Digest Window, the selected fragment is highlighted both on the left side (the text) and in the Gel (right) side. PROTEIN TRANSLATION AND OPEN READING FRAMES 111) If you would like, you can see the amino acid translation of the consensus in all reading frames. In the Aligned Reads Window, push down the left mouse button on the 'Misc' menu and release on 'Show Top Strand Protein Translation'. Try again but this time release on 'Show Bottom Strand Protein Translation'. Notice that there are 2 characters that are in magenta color. What are those characters? Why are they made in a different color? To not show the protein translation, push down the left mouse button on the 'Misc' menu and release on 'Don't show protein translation'. 112) You can search for open reading frames (a methionine and a stop codon within the same reading frame) within a contig. In the Aligned Reads Window, push the left mouse button on 'Navigate' and release on 'Search for Open Reading Frames'. Notice that the open reading frames are shown for all 6 reading frames and are sorted by length. ERROR RATE 113) In the Aligned Reads Window is a box (upper right) labelled 'Err/10kb'. This is the estimated error rate for this contig, and it is a good indicator of when you are done (or not done) finishing. In addition, you can find the error rate for a particular region of contig as follows: Point at 'Misc' menu, hold down the left mouse button, pull down and release on 'Show Error Info For Region'. Fill in the boxes for left and right consensus position, click on 'Calculate' and you will be given the error and single subclone data for that region. RUNNING PHRED and PHRAP phred and phrap *must* be run via the phredPhrap perl script. If you don't do this, you are on your own. If you run phred on its own, and then you run phrap on its own, you will get an ace file that will not be usable by Consed. After you have run into problems (and you probably will), then do not email us--instead please use the phredPhrap script. To use the phredPhrap script to run phred and phrap: 114) Type: phredPhrap -V It should say: 030326 (or newer). If it does not, then you probably have not installed all the perl scripts from the scripts directory, as directed in INSTALLING CONSED. 115) Make a copy of the standard dataset. E.g., (First go up by typing "cd .." until you see "standard" when you type "ls". Then type: cp -r standard test cd test 116) Delete all the files in phd_dir and edit_dir: rm phd_dir/* rm edit_dir/* 117) cd edit_dir 118) Run phredPhrap by typing phredPhrap That's it--you no longer need to type *any* arguments, and generally you should not. If you want to add phrap options, you can do that: e.g., phredPhrap -forcelevel 3 Then run Consed on the resulting ace file as indicated in the beginning of the Quick Tour (above). If you have any problems, this is the time to diagnose them before you use your own data. COMMON PROBLEMS RUNNING PHREDPHRAP 119) Problems were due to polyphred. To check this, in phredPhrap, leave the following line: $bUsingPolyPhred = 0; This will make polyphred not be used. If the problem then goes away, you will know the problem has something to do with polyphred so do not contact any of the phred/phrap/Consed people. Instead, contact the polyphred people: http://droog.mbt.washington.edu and dpc@u.washington.edu and debnick@u.washington.edu 120) Permission problems. Check that you have write access to the phd_dir and edit_dir directories. You can do this by trying to create a file in those directories: touch ../phd_dir/xxx which creates a file ls -l ../phd_dir/xxx which checks if the file was created. Do the same with ../edit_dir/xxx If you get a permission problem, do not contact me. UNIX permission problems are very simple for anyone who knows UNIX--get someone locally who understands UNIX and can help you solve the permission problem. ---------------------------------------------------------------------------- WHAT IS AUTOFINISH? Autofinish automatically chooses reads for finishing. Autofinish sometimes is able to completely finish a project with no human decisions. In other cases Autofinish mostly finishes a project, and a human just needs to do the final difficult problems since all the routine problems have already been completed by Autofinish. Thus a human finisher is able to complete far more projects in the same length of time. Autofinish is flexible to the finishing strategy of your lab. It can be used to finish with just universal primer reads, just oligo walks, just minilibraries, or a combination of these. It can be used to finish either genomic or cDNA. Autofinish will do the following: -close gaps -improve sequence quality -determine the relative orientation of contigs -ensure that, at each consensus base, at least 2 reads from different templates are aligned (You can configure Autofinish to do any combination of these tasks.) Autofinish will suggest the following types of experiments: -universal primer reads (forward or reverse) -custom primer reads with subclone templates -custom primer reads with whole clone templates -minilibraries (transposon or shatter) from subclone templates -PCR (You can configure Autofinish to suggestion any combination of these experiments.) ------------------------------------------------------------------------ USING AUTOFINISH Note: Before you use Autofinish on your own data, you must modify determineReadTypes.perl. See INSTALLING CONSED above for information about this. To do the exercises in this section, it would help to be able to edit a file under UNIX and run a program under UNIX. If you can't do that, have someone teach you. (It will not work to edit a file on Windows and then transfer to UNIX.) Typical editors on UNIX are vi and emacs, but pico is probably the simplest for occasional users. You can find more information on pico from: http://www.strath.ac.uk/IT/Docs/IntroToUnix/node122.html You should also learn how to examine a file in UNIX, how to move around the filesystem, etc. If you don't know how to do this, consult: http://www.washington.edu/computing/unix/startdoc/files.html and http://www.washington.edu/computing/unix/startdoc/directories.html There are also many books about Unix at bookstores. 121) Type: cd autofinish/edit_dir (You might need to first type "cd ../.." depending on where you are.) 122) Try starting Autofinish by typing: ../../consed -ace autofinish.fasta.screen.ace.1 -autofinish (Note 'consed' above may be 'consed_solaris', 'consed_solaris_intel', 'consed_solaris64', 'consed_alpha', 'consed_hp', 'consed_sgi', 'consed_ibm', 'consed_mac', 'consed_linux2.4', 'consed_linux2.6', 'consed_linux2.6_dyn', 'consed_amd64', or 'consed_amd64_dyn', depending on your executable. If you have trouble, use that 'ls' command (see above) or consult the person who installed Consed! ) If Autofinish says: Run-time exception error; current exception: InputDataError No handler for exception. Abort that means that you have not followed the instructions under 'INSTALLING CONSED' above. Please follow those instructions and then try this again. When you have successfully run the above command, Autofinish will create 7 files: autofinish.fof (project name).001014.155627.customPrimers (project name).001014.155627.nav (project name).001014.155627.out (project name).001014.155627.sorted (project name).001014.155627.univForwards (project name).001014.155627.univReverses Where '001014.155627' is replaced by your current date and time in format YYMMDD.HHMISS. The first file, autofinish.fof, is a file of filenames. It contains the names of the other files. (project name).001014.155627.univForwards is the summary file of the suggested universal forward subclone reads (project name).001014.155627.univReverses is the summary file of the suggested universal reverse subclone reads (project name).001014.155627.customPrimers is the summary file of the suggested custom primer reads These are the files you will typically use for directing your bench work. If you like, you can import these files into Excel since the fields are separated by commas. The .out file is the Autofinish output file. This is the most important file to examine while you are evaluating Autofinish. If you want to know *why* Autofinish picked the reads it did, it will tell you. Consult this file before you start complaining about Autofinish's choices. I've had people complain, and then, once they look in the .out file (*not* any of the other files), they learn information that persuades them that Autofinish was correct all along. This is hard to over-emphasize, but I will try to over-emphasize it: It will tell you lots more, such as the orientation of the contigs. It will also tell you the value of all Autofinish parameters used. If you try to customize one of the parameters, check in the .out file to be sure that Autofinish used the value you intended. CONSULT THE .out FILE CAREFULLY IF YOU DISAGREE WITH ANY OF AUTOFINISH'S CHOICES! The .sorted file gives the reads sorted by contig and position. This file is useful if you want to find what reads Autofinish suggested for a particular location. It is *not* useful for understanding *why* Consed chose a particular read. It is deliberately terse to make it useful for automation the ordering of reads. The .nav file is a custom navigation file (see "CUSTOM NAVIGATION" far below). This file allows a Consed user to just click 'next', 'next', ... to review all of Autofinish's suggestions in context. This is a great way to quickly and easily review all of the reads suggested by Autofinish. This finishing tool is designed to be run in batch after each assembly. In a high throughput operation, the production people can make these reads without anyone using Consed to examine the assembly interactively. Only when Autofinish cannot help you any longer (generally after 3 or more times of running Autofinish, making the reads, and re-assembling), must you bring up Consed graphically and examine the assembly. We suggest that you write some of your own software to parse the summary files to automatically order primers and reads. The summary files (.customPrimers, .univForwards, .univReverses) will not change much but the .out file may change, so don't try to parse it. AUTOFINISH: MINIMUM NUMBER OF ERRORS FIXED PER READ 123) By default, the minimum number of errors fixed by an experiment is 0.02 Human finishers typically look for low consensus quality regions--regions that have one or more bases below a particular quality threshold. However, Autofinish can do better: it can find regions where the *total* number of errors is greater than some particular cutoff value. This method can find regions where none of the bases are low quality, but many are medium quality and thus the total number of errors in the region is high. Autofinish will also ignore regions that have a very few low quality bases, as long as the total number of errors is smaller than your cutoff. This is a better critereon because it is the total number of errors that you are trying to reduce when finishing--not the number of bases with quality below some arbitrary cutoff. Two bases of quality 20 have 0.02 errors (on average). Similarly, 20 bases of quality 30 have 0.02 errors (on average). (Quality values were explained at the beginning of this document.) Suppose that you want Autofinish to suggest an additional read for an area that even just has one quality 20 base. (Be aware that Autofinish will consider 10 quality 30 bases to be just as severe as 1 quality 20 base since, on average, they will both have precisely the same number of errors: 0.01) 124) EDIT PARAMETERS: HOW TO CHANGE CONSED/AUTOFINISH PARAMETERS This shows how to change consed.autoFinishMinNumberOfErrorsFixedByAnExp. To change any other parameter, follow these same instructions replacing consed.autoFinishMinNumberOfErrorsFixedByAnExp with the parameter you want to change. In the edit_dir directory is a file called ".consedrc" which you will only see if you use "ls -a" instead of just "ls". In that .consedrc, add the following line: consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.01 You can do this using an editor, such as pico, or you can do it with Consed. To do it with Consed, bring up consed as follows: consed -ace autofinish.fasta.screen.ace.1 On the Consed Main Window, point to the "Options" menu, push down the left mouse button and release on "Edit Consed/Autofinish Parameters". Up will pop the "Edit Parameters" window. Near the top is "consed.autoFinishMinNumberOfErrorsFixedByAnExp". Point and click in the box on the left containing 0.02 just underneath "consed.autoFinishMinNumberOfErrorsFixedByAnExp". After clicking, the box outline should turn bold and the cursor should start blinking. Change the 0.02 to 0.01. Click on "just project" near the bottom of the window. The box containing 0.01 should turn red indicating that it is now different than the default. Then click "save". A box titled "Name of parameter file to write" should pop up. Click "ok". Note: you can changed more than one of these values before clicking "save". When using the Edit Parameters Window, I suggest that you do not click on the up and down arrows of the vertical scrollbar because these will scroll by too much. Instead, I suggest you point to the thumb of the vertical slider, hold down the left mouse button and drag the thumb. Alternatively, point to the black space above or below the thumb and click with the left mouse button. You need to try this to understand. To be sure that everything happened correctly, look at .consedrc file. It should contain the line: consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.01 (If you don't know how to view a file, get a UNIX book and learn the commands "less", "more", "pico", "vi", or "emacs".) (Get in the habit of checking .consedrc after using Consed's Edit Parameter Window.) AUTOFINISH: MINIMUM NUMBER OF ERRORS FIXED PER READ (continued) Then run Autofinish again: consed -ace autofinish.fasta.screen.ace.1 -autofinish Look at the files just created by typing 'ls -tlr' and look at the .out file by bringing it up with your favorite UNIX editor. You should see: PARAMETERS_CHANGED_FROM_DEFAULTS { . consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.010 . . Further down is a section: PARAMETERS { ! If you want to modify any of these parameters, just cut/paste ! the relevant line into your ~/.consedrc file ! (or into the edit_dir/.consedrc file) ! In the following, I have annotated the parameters with the following ! symbols: ! ! (YES) freely customize to your own site ! (OK) don't change unless you have a specific need and know what you ! are doing ! (NO) don't change this! This section contains all Autofinish parameters, whether you have changed them or not. Thus a changed parameter will be in both lists. Find consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.010 in this second list. Then compare the .sorted files from this run of Autofinish and the previous run of Autofinish in which the consed.autoFinishMinNumberOfErrorsFixedByAnExp value 0.02 You will notice that there are 2 additional reads suggested when the parameter is 0.01. There is a resequence with dye terminator chemistry of the djs228_474 template and a de novo reverse on template djs228_2632. Look at the .out file to see why Autofinish chose these reads. It will indicate that the first read is mainly to fix 0.01 errors in the region from 2536 to 2545 and the second read to mainly fix 0.01 errors from 969 to 978. Bring up Consed to see what is in the 10 base region from 2536 to 2545. You will see that there is a quality 25 base at 2539 and a quality 21 base at 2540. After that come some bases whose qualities are in the high 30s. In the Aligned Reads Window, point at the Misc menu, hold down the left mouse button, and release on Show Error for a Region. Enter 2539 and 2549 for the "Left Consensus Position of Region" and "Right Consensus Position of Region" respectively and click on "Calculate". You will see that there are .0135 errors in this region. This is less than 0.02 so Autofinish will not try to fix this region unless you reduce consed.autoFinishMinNumberOfErrorsFixedByAnExp to 0.01 The default is 0.02 because most labs do not want to fix regions that have less than 0.02 errors. 125) DIVERSION: UNIX LESSON Note for UNIX novices: Earlier, I said that you only needed to know 3 UNIX commands: pwd, ls, and cd. Then I added "ls -a", "less" and an editor (such as pico). Now I want you to learn one more: ls -tlr This is the same as ls, but it puts one file on a line and prints the lines so that the most recent files are on the bottom. Since you will be creating many, many files as you work through these Autofinish exercises, this command gives an easy way to see the files you have just created, without having to always look at autofinish.fof to look for the names of the files you just created. AUTOFINISH: CHANGING MELTING TEMPERATURES 126) Use 'ls -tlr' to find the most recent .out file. Search in the .out file (using your favorite editor) for MeltingTemp and you will find the following lines: consed.primersMinMeltingTemp: 55 consed.primersMaxMeltingTemp: 60 Some labs prefer to use primers with lower melting temperatures. In your .consedrc file, put the following lines: consed.primersMinMeltingTemp: 50 consed.primersMaxMeltingTemp: 55 You can do this by following the instructions above under HOW TO CHANGE CONSED/AUTOFINISH PARAMETERS. When you are done doing that, look in the .consedrc file to make sure it contains the above 2 lines. Then run Autofinish again: consed -ace autofinish.fasta.screen.ace.1 -autofinish Using your favorite editor, check that the .out file you just created says: consed.primersMinMeltingTemp: 50 consed.primersMaxMeltingTemp: 55 (You can find the most recent .out file by typing 'ls -tlr'.) Compare the .sorted files from this run of Autofinish and the previous run. The difference should be the custom primer read: The previous .sorted file had: tcttttgtctttccatatacatttt,56 which means the melting temperature is 56. The latest .sorted file had: cattttagaatcagtttgttg,50 which means the melting temperature is 50. 127) AUTOFINISH: JUST CLOSING GAPS You could use Autofinish to just close gaps (you are not interested in fixing single subclone regions or weak regions). Add the following to the .consedrc file (and remove everything else so that Autofinish uses the default values for everything else): consed.autoFinishCoverLowConsensusQualityRegions: false consed.autoFinishCoverSingleSubcloneRegions: false If you are using the Edit Parameter Window to change these values, you will find them when scrolling about 1/3 way down. Change the consed.primersMinMeltingTemp and consed.primersMaxMeltingTemp back to their original values. Then check the .consedrc to make sure it contains the above 2 lines. (Get in the habit of checking .consedrc after using Consed's Edit Parameter Window.) Now you should see in the .sorted file just 4 reads: one custom primer read pointing out the left end of the contig and 3 reverses off the left end of the contig. The right end is not extended because Autofinish recognizes that it is the end of the BAC. You can change any of the parameters listed at the top of the Autofinish output file (or actually any of the more exhaustive list of parameters listed in the 'Info' menu, 'Show Consed Parameters' list.) We believe the defaults are an excellent starting point. 128) AUTOFINISH: JUST CLOSING GAPS JUST USING WALKS One high-throughput operation was only interested in closing gaps and only interested in using walks to close those gaps. This is the appropriate set of Autofinish parameters to do this: consed.autoFinishCoverSingleSubcloneRegions: false consed.autoFinishCoverLowConsensusQualityRegions: false consed.autoFinishAllowDeNovoUniversalPrimerSubcloneReads: false consed.autoFinishAllowPCR: false consed.autoFinishAllowResequencingReads: false consed.autoFinishAllowMinilibraries: false consed.autoFinishNearGapsSuggestEachMissingReadOfReadPairs: false consed.autoFinishCallReversesToFlankGaps: false (and every other parameter left the default value). The first 2 parameters are the same as the "AUTOFINISH: JUST CLOSING GAPS" section (above). The other parameters tell Autofinish all of the types of reactions it is not allowed to use, leaving just walks. Try this. Now you should see only a single read, a walk, pointing left off the left end of the contig. 129) AUTOFINISH: NOT REPEATING FAILED EXPERIMENTS For this exercise, keep a backup copy of the ace file: cp autofinish.fasta.screen.ace.1 autofinish.fasta.screen.ace.1.save If you run Autofinish with the -doExperiments parameter (see below), -doExperiments causes Autofinish to record its suggestions in the ace file (hence changing the ace file). If one of these suggested reads fails to fix a problem, when Autofinish is run again it won't pick the same read again. consed -ace (ace file name) -autofinish -doExperiments If a forward or reverse universal primer read failed, Autofinish (when run in a subsequent round) will not suggest that same experiment. If a custom primer read fails, Autofinish will not pick that same experiment again, and it won't pick a custom primer read that is even close to the failed one. 'Close' is defined by the parameter: consed.autoFinishNewCustomPrimerReadThisFarFromOldCustomPrimerRead: 50 In addition, Autofinish (the next time it is run) will tell you how well each experiment did in solving the problem it was intended to solve. Return the parameters to the defaults and try this by running Autofinish twice like this: consed -ace autofinish.fasta.screen.ace.1 -autofinish -doExperiments consed -ace autofinish.fasta.screen.ace.1 -autofinish -doExperiments and look at the .out file from the 2nd run. (You can find the most recent .out file by typing 'ls -tlr'.) You should see lines such as this: rejecting experiment: reverse universal primer read with template djs228_1094 because an earlier round of autofinish called this with expid: 1 rejecting experiment: reverse universal primer read with template djs228_1422 because an earlier round of autofinish called this with expid: 2 rejecting experiment: reverse universal primer read with template djs228_1034 because an earlier round of autofinish called this with expid: 3 This is Autofinish trying experiment after experiment but finding they were already suggested in an earlier round of Autofinish. You should not type '-doExperiments' if you do not intend to do the experiments Autofinish suggests. If you use -doExperiments, but you don't really do the experiments, and then you run Autofinish again, Autofinish will be very upset--it will think that all of its suggested experiments failed (because it can't find them). It will see that all of the problems are still present but it will think that it should not choose any of those same experiments again so it will suggest different experiments that will not be as good as its original suggestions. -doExperiments will also cause suggested oligos to be tagged. Primer id's created by Autofinish use the same naming scheme as primers created in Consed and they will not conflict with each other. For example, if Autofinish creates oligos djs14.1, djs14.2, and djs14.3, then the next primer that a user accepts will be djs14.4. If Autofinish is run a second time, it will start with primer djs14.5. When you have completed this exercise with -doExperiments, replace the original .ace file by typing: cp autofinish.fasta.screen.ace.1.save autofinish.fasta.screen.ace.1 130) AUTOFINISH: doNotFinish particular regions If there is a region that you don't care to finish (e.g., it has already been finished by an overlapping clone or you know there is no gene there), then you can put a doNotFinish tag on the consensus and Autofinish will not try to finish this area. First, delete the .consedrc file (or, if you are using the Edit Parameter Window of Consed, restore the parameters to their default values) and run Autofinish again: consed -ace autofinish.fasta.screen.ace.1 -autofinish Bring up consed: consed -ace autofinish.fasta.screen.ace.1 and put a doNotFinish tag on the region from 2000 to 4000. (If you don't know how to do that, read through the Consed Quick Tour, above.) Save the assembly as autofinish.fasta.screen.ace.2 Run Autofinish again: consed -ace autofinish.fasta.screen.ace.2 -autofinish Look at the .out files for each of the 2 runs of Autofinish. (You can find the most recent .out files by typing 'ls -tlr'.) You will notice in the .out file for the 2nd run of Autofinish that, in the other than the experiments to extend the contig to the left, there is only one experiment which is from 315 to 1662. If you find that experiment in the .out file, it will say "Contig1 0.05 errors fixed in region from 315 to 1662 fixing 0.05 errors from 969 to 978" The "969 to 978" gives the worst 10 base window that the read is intended to fix. If you look with Consed, you will see that there is a quality 12 base at 974. You can also use doNotFinish tags to prevent Autofinish from *extending* a contig into a gap by putting a doNotFinish tag near the end of the contig and setting the following Autofinish parameters: consed.autoFinishDoNotExtendContigsWhereTheseTagsAre: doNotFinish consed.autoFinishDoNotExtendContigsIfTagsAreThisCloseToContigEnd: 50 131) AUTOFINISH: NOT USING PARTICULAR SUBCLONE TEMPLATES If you no longer have a template that was used in shotgun, and thus you don't want Autofinish to pick that template, you can put it in a file badTemplates.txt in edit_dir. This is a simple file with one name per line. Using your favorite UNIX editor, create a file called "badTemplates.txt" in edit_dir. Make it contain a single line: djs228_1094 Delete .consedrc (or, if you are using the Edit Parameter Window, restore the parameters to their defaults) and run autofinish again: consed -ace autofinish.fasta.screen.ace.1 -autofinish Search the .out file for djs228_1094. You will find one line like this: not using template: djs228_1094 because in bad templates file Now try deleting badTemplates.txt and running autofinish again the same way. You will notice there are many differences in reads chosen, since djs22_1094 is now available again for making reverses as well as a template for custom primer walks. badTemplates.txt can accept "*" (match any characters) as part of the name. For example, djs140_23* will eliminate templates: djs140_235684 djs140_235783 djs140_2326 etc. 132) AUTOFINISH: NOT USING ENTIRE LIBRARIES FOR FINISHING In addition to the badTemplates.txt file, you can use a badLibraries.txt file which contains a list of all libraries that are off-limits to Autofinish (e.g., you threw away all subclone templates from this library or they are from a different lab which gave you the chromatograms but not the templates). Autofinish determines the library of a read by the following in the PHD file: WR{ template dscript 990603:090231 name: djs366_101 lib: library1 } where "library1" is replaced by the actual library name. Take a look at any phd file in autofinish/phd_dir and you will see this. Generally, determineReadTypes.perl puts this library information into the PHD file. Make sure that badTemplates.txt is deleted and .consedrc is either deleted (or use the Edit Parameter Window to restore the defaults) and run Autofinish again. consed -ace autofinish.fasta.screen.ace.1 -autofinish Now create a file badLibraries.txt containing a single line: lib1 and run autofinish again: consed -ace autofinish.fasta.screen.ace.1 -autofinish Look at the .out file. You will see lines like this: not using template: djs228_1034 because in bad libraries file not using template: djs228_1051 because in bad libraries file not using template: djs228_1094 because in bad libraries file . . . You will see that there are no reads suggested that use any of these templates, even though some of them (e.g.., djs228_1034) were used in the Autofinish run (above) before you created the badLibraries.txt file. When you start doing this with your own data, you must put the lib: line into your phd files. Do this by modifying determineReadTypes.perl. 133) MULTIPLE LIBRARIES WITH DIFFERENT INSERT SIZES If different libraries have different insert sizes, Autofinish must know the insert size of each library. If there are 5 or more forward-reverse pairs, where the forward and reverse are both in the same contig, then Consed/Autofinish calculates the insert size of the library by finding the mean and standard deviation of the insert sizes of these forward-reverse pairs. The maximum insert size of the library is set at the mean plus 2.5 times the standard deviation. If there are fewer than 5 forward-reverse pairs, where the forward and reverse are both in the same contig, Consed/Autofinish considers this statistical information unreliable so instead relies on a file called 'librariesInfo.txt" which must be placed in edit_dir (where the ace file is). This file looks like this: LIB{ name: lib0 avgInsertSize: 1500 maxInsertSize: 3000 stranded: double cost: 600.0 } LIB{ name: lib1 avgInsertSize: 3000 maxInsertSize: 5000 stranded: double cost: 1000.0 } LIB{ name: lib2 avgInsertSize: 10000 maxInsertSize: 12000 stranded: double cost: 5000.0 } 'name' is the name of the library. This is the name that goes into the PHD file after the 'lib:' keyword (see AUTOFINISH: NOT USING ENTIRE LIBRARIES FOR FINISHING above). 'avgInsertSize' is the average insert size of the library--the figure to be used by Autofinish if there are not enough forward/reverse pairs for Autofinish to calculate the mean insert size of the library. 'maxInsertSize' is the maximum insert size--if forward/reverse pairs are further apart than this, Autofinish will assume these reads are misassembled. 'stranded' is whether this template is single or double stranded. 'cost' is the cost of making a minilibrary out of a template from this library. In .consedrc, there must be a line like this: consed.primersMaxInsertSizeOfASubclone: 5000 where 5000 is replaced by whatever the maximum insert size of all of your different libraries. For this exercise make .consedrc have a single line: consed.primersMaxInsertSizeOfASubclone: 12000 Alternatively, use the Edit Parameter Window to set consed.primersMaxInsertSizeOfASubclone to 12000. For this exercise I have a file in edit_dir called "librariesInfo.txt_hide". To make Autofinish pay attention to it, do the following: cp librariesInfo.txt_hide librariesInfo.txt Delete badLibraries.txt: rm badLibraries.txt Before you run Autofinish again, first restart Consed: consed -ace autofinish.fasta.screen.ace.1 On Consed's Main Window, point to 'Info', hold down the left mouse button, and release on 'Show Library Info'. You should see the names of your libraries and the correct number of reads in each library. This feature will be useful in debugging your use of librariesInfo.txt Then run Autofinish again: consed -ace autofinish.fasta.screen.ace.1 -autofinish Look at the .out file. Look for the following: "Choosing de novo universal primer reads to try to close gaps" You will see there are many reads under this heading. These are the lib1 and lib2 reads that have a large average insert size and thus span the gap. Autofinish did not choose some of these reads before because, if the insert size were only 1500 bases, these reads would not have helped to close the gap. When you are done with this exercise, delete librariesInfo.txt and .consedrc When there are many reads from the same library, Consed/Autofinish will look at the forward/reverse pairs that are within the same contig (so the insert size of that template can be directly measured) and figure out the mean and standard deviation of the insert size of templates from that library. Consed/Autofinish will use these numbers rather than the number from librariesInfo.txt 134) AUTOFINISH CLOSING GAPS WITH MINILIBRARIES If you wanted Autofinish to *only* suggest minilibraries to close gaps, use the following parameters: consed.autoFinishAllowWholeCloneReads: false consed.autoFinishAllowCustomPrimerSubcloneReads: false consed.autoFinishAllowResequencingReads: false consed.autoFinishAllowDeNovoUniversalPrimerSubcloneReads: false consed.autoFinishAllowPCR: false consed.autoFinishAllowResequencingAUniversalPrimerAutofinishRead: false consed.autoFinishCallReversesToFlankGaps: false consed.autoFinishAllowMinilibraries: true consed.autoFinishAlwaysCloseGapsUsingMinilibraries: true consed.autoFinishPrintMinilibrariesSummaryFile: true For this exercise, type: cd assembly_view/edit_dir (You might need to first type "cd ../.." depending on where you are.) Attention! This is *not* the same directory you have been using. It, autofinish/edit_dir, does not have any gaps so it cannot be used for this exercise. Create a .consedrc file with the parameters above in it. Alternatively, start Consed in this directory and use the Edit Parameter Window to modify the parameters as above. Then run Autofinish: consed -ace assembly_view.fasta.screen.ace.1 -autofinish When it has completed, look in the .out file. You will see the following: Enough existing fwd/rev pairs to establish: Left end of Contig3 has 13 fwd/rev pairs connecting it to Right end of Contig2 with gap size -460 (contigs overlap) Trying to suggest minilibrary for gap between right end of Contig2 and left end of Contig3 MINILIBRARY{ best template: djs736a2_fp04q274 from lib djs736a2 size: 3607 errors fixed: 0.01 errors fixed per dollar: 0.00 connecting right end of Contig2 to left end of Contig3 with estimated gap size -460 alternative template: djs736a1_fp02q472 from lib dj