{"__v":73,"_id":"5735936aafab441700723a56","category":{"__v":0,"_id":"5735936aafab441700723a54","project":"5735936aafab441700723a50","version":"5735936aafab441700723a53","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-05-13T08:42:18.644Z","from_sync":false,"order":0,"slug":"documentation","title":"Get Started"},"parentDoc":null,"project":"5735936aafab441700723a50","user":"573592b84b0ab120000b7d44","version":{"__v":12,"_id":"5735936aafab441700723a53","project":"5735936aafab441700723a50","createdAt":"2016-05-13T08:42:18.615Z","releaseDate":"2016-05-13T08:42:18.615Z","categories":["5735936aafab441700723a54","5735a32931a73b1700887c94","5735b55beceb872200abbc6c","5735b56eb667601700d3bd6f","5735b9ba4b0ab120000b7dd4","5735b9c94b0ab120000b7dd5","5735cb131f16241700c8a0f7","5735e5c4e4824c3400aa1f21","5735e5d9e4824c3400aa1f23","5735e5f2ec67f6290013ac72","573ecfe0804f901700a9dfc7","573f276c7eeb8b190094ca7d"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":false,"codename":"","version_clean":"1.0.0","version":"1.0"},"updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-05-13T08:42:18.691Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":0,"body":"[PeerJ Preprints 4:e2401v1](https://doi.org/10.7287/peerj.preprints.2401v1)\n\n[block:callout]\n{\n  \"type\": \"info\",\n  \"title\": \"Dockerised setup\",\n  \"body\": \"We have incorporated easy-import into to a Dockerised custom Ensembl setup at [GenomeHubs.org](http://genomehubs.org) and this documentation will be fully updated to reflect this approach soon.\"\n}\n[/block]\n\n[block:callout]\n{\n  \"type\": \"success\",\n  \"title\": \"Very quick start...\",\n  \"body\": \"Install docker then:\\n\\n```\\ncd\\ngit clone https://github.com/genomehubs/demo\\ndemo/demo.sh\\n```\\n\\n(best run as UID 1000)\"\n}\n[/block]\nThe instructions below will help you get an Ensembl database and website up and running in an afternoon - with four Lepidopteran genomes mirrored from [Ensembl Metazoa](http://metazoa.ensembl.org) plus a fresh import of the genome of the winter moth *Operophtera brumata* direct from publicly hosted ``.gff`` and ``.fasta`` files.  \n\nIf you are more interested in importing your own data it's probably still a good idea to check that this example works, then the rest of the documentation will help you get to know the configuration options available and how to customise the ``.ini`` files to suit your own data.  The sidebar links also provide more detailed descriptions of each of the steps below, and a summary of the relevant configuration options.\n\nIf you just want to setup a local [Ensembl](http://ensembl.org) mirror with existing databases then check out [easy mirror](https://github.com/lepbase/easy-mirror), which is included in easy import as a submodule.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/0d9706e-easy-import.png\",\n        \"easy-import.png\",\n        2000,\n        1125,\n        \"#2b7027\"\n      ],\n      \"sizing\": \"smart\",\n      \"border\": false,\n      \"caption\": \"An overview of Ensembl setup with the easy-import pipeline\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Stage 1 - Server/database configuration\"\n}\n[/block]\nUsing ``setup.ini``/``setup-db.ini`` will setup an [Ensembl Metazoa](metazoa.ensembl.org) mirror with four Lepidopteran species.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"sudo apt-get update\\nsudo apt-get upgrade\\nsudo apt-get install git\\ncd ~\\ngit clone --recursive https://github.com/lepbase/easy-import ei\\ncd ~/ei/em\\nsudo ./install-dependencies.sh ../conf/setup.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 1.1: Install dependencies\"\n    }\n  ]\n}\n[/block]\nThis first step requires root permissions and assumes a fresh install of Ubuntu 14.04 so you might like to [read more](doc:step-11-install-dependencies) about what it is doing before running the commands.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/ei/em\\n./setup-databases.sh ../conf/setup-db.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 1.2: Setup database connections\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-12-setup-database-connections)\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/ei/em\\n./update-ensembl-code.sh ../conf/setup.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 1.3: git clone ensembl repositories\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-13-git-clone-ensembl-repositories)\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Stage 2 - Core import\"\n}\n[/block]\nUsing ``core-import.ini`` will install a new core database for the winter moth *Operophtera brumata*\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"mkdir ~/import\\ncd ~/import\\nperl ../ei/core/summarise_files.pl ../ei/conf/core-import.ini\",\n      \"language\": \"shell\",\n      \"name\": \"(optional) Step 2.1: Fetch/summarise assembly/annotation files\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-21-fetchsummarise-assemblyannotation-files) \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/import\\nperl ../ei/core/import_sequences.pl ../ei/conf/core-import.ini\\nperl ../ei/core/import_sequence_synonyms.pl ../ei/conf/core-import.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 2.2: Create database and load sequence data\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-22-create-database-and-load-sequence-data)\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/import\\nperl ../ei/core/prepare_gff.pl ../ei/conf/core-import.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 2.3: Prepare the gff file for import\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-23-prepare-the-gff-file-for-import) \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/import\\nperl ../ei/core/import_gene_models.pl ../ei/conf/core-import.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 2.4: Import gff from prepared file\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-24-import-gff-from-prepared-file) \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/import\\nperl ../ei/core/verify_translations.pl ../ei/conf/core-import.ini\",\n      \"language\": \"shell\",\n      \"name\": \"(optional) Step 2.5: Verify import\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-25-verify-import) \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/import\\nperl ../ei/core/import_blastp.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini\\nperl ../ei/core/import_repeatmasker.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini\\nperl ../ei/core/import_interproscan.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini\\nperl ../ei/core/import_cegma_busco.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini\",\n      \"language\": \"shell\",\n      \"name\": \"(optional) Step 2.6: Import additional annotations\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-26-import-additional-annotations) \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/import\\nperl ../ei/core/export_sequences.pl ../ei/conf/core-import.ini\\nperl ../ei/core/export_json.pl ../ei/conf/core-import.ini\",\n      \"language\": \"shell\",\n      \"name\": \"(optional) Step 2.7: Export files\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-27-export-files) \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/import\\nperl ../ei/core/index_database.pl ../ei/conf/core-import.ini\",\n      \"language\": \"shell\",\n      \"name\": \"(optional) Step 2.8: Generate search index\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-28-generate-search-index) \n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Stage 3 - Web site configuration\"\n}\n[/block]\nedit ``setup.ini`` to add ``operophtera_brumata_v1_core_31_84_1`` to ``[DATA_SOURCE] SPECIES_DBS``\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/ei/em\\n./update-ensembl-code.sh ../conf/setup.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 3.1: Update Ensembl webcode\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-31-update-ensembl-webcode) \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"cd ~/ei/em\\n./reload-ensembl-site.sh ../conf/setup.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Step 3.2: Reload Ensembl website\"\n    }\n  ]\n}\n[/block]\n[...read more](doc:step-32-reload-ensembl-website)\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"(optional) Stage 4 - Compara import\"\n}\n[/block]\nSetting up a compara requires the results of a set of analyses in addition to a ``.ini`` file similar to ``compara-import.ini`` (which contains the configuration used for the [lepbase.org](http://lepbase.org) compara).  Once you have the appropriate files listed in [Stage 4 requirements](doc:stage-4-requirements), you will be ready to start importing a a compara.","excerpt":"The easiest way to start importing data into an Ensembl","slug":"quick-start-guide","type":"basic","title":"Quick start guide"}

Quick start guide

The easiest way to start importing data into an Ensembl

[PeerJ Preprints 4:e2401v1](https://doi.org/10.7287/peerj.preprints.2401v1) [block:callout] { "type": "info", "title": "Dockerised setup", "body": "We have incorporated easy-import into to a Dockerised custom Ensembl setup at [GenomeHubs.org](http://genomehubs.org) and this documentation will be fully updated to reflect this approach soon." } [/block] [block:callout] { "type": "success", "title": "Very quick start...", "body": "Install docker then:\n\n```\ncd\ngit clone https://github.com/genomehubs/demo\ndemo/demo.sh\n```\n\n(best run as UID 1000)" } [/block] The instructions below will help you get an Ensembl database and website up and running in an afternoon - with four Lepidopteran genomes mirrored from [Ensembl Metazoa](http://metazoa.ensembl.org) plus a fresh import of the genome of the winter moth *Operophtera brumata* direct from publicly hosted ``.gff`` and ``.fasta`` files. If you are more interested in importing your own data it's probably still a good idea to check that this example works, then the rest of the documentation will help you get to know the configuration options available and how to customise the ``.ini`` files to suit your own data. The sidebar links also provide more detailed descriptions of each of the steps below, and a summary of the relevant configuration options. If you just want to setup a local [Ensembl](http://ensembl.org) mirror with existing databases then check out [easy mirror](https://github.com/lepbase/easy-mirror), which is included in easy import as a submodule. [block:image] { "images": [ { "image": [ "https://files.readme.io/0d9706e-easy-import.png", "easy-import.png", 2000, 1125, "#2b7027" ], "sizing": "smart", "border": false, "caption": "An overview of Ensembl setup with the easy-import pipeline" } ] } [/block] [block:api-header] { "type": "basic", "title": "Stage 1 - Server/database configuration" } [/block] Using ``setup.ini``/``setup-db.ini`` will setup an [Ensembl Metazoa](metazoa.ensembl.org) mirror with four Lepidopteran species. [block:code] { "codes": [ { "code": "sudo apt-get update\nsudo apt-get upgrade\nsudo apt-get install git\ncd ~\ngit clone --recursive https://github.com/lepbase/easy-import ei\ncd ~/ei/em\nsudo ./install-dependencies.sh ../conf/setup.ini", "language": "shell", "name": "Step 1.1: Install dependencies" } ] } [/block] This first step requires root permissions and assumes a fresh install of Ubuntu 14.04 so you might like to [read more](doc:step-11-install-dependencies) about what it is doing before running the commands. [block:code] { "codes": [ { "code": "cd ~/ei/em\n./setup-databases.sh ../conf/setup-db.ini", "language": "shell", "name": "Step 1.2: Setup database connections" } ] } [/block] [...read more](doc:step-12-setup-database-connections) [block:code] { "codes": [ { "code": "cd ~/ei/em\n./update-ensembl-code.sh ../conf/setup.ini", "language": "shell", "name": "Step 1.3: git clone ensembl repositories" } ] } [/block] [...read more](doc:step-13-git-clone-ensembl-repositories) [block:api-header] { "type": "basic", "title": "Stage 2 - Core import" } [/block] Using ``core-import.ini`` will install a new core database for the winter moth *Operophtera brumata* [block:code] { "codes": [ { "code": "mkdir ~/import\ncd ~/import\nperl ../ei/core/summarise_files.pl ../ei/conf/core-import.ini", "language": "shell", "name": "(optional) Step 2.1: Fetch/summarise assembly/annotation files" } ] } [/block] [...read more](doc:step-21-fetchsummarise-assemblyannotation-files) [block:code] { "codes": [ { "code": "cd ~/import\nperl ../ei/core/import_sequences.pl ../ei/conf/core-import.ini\nperl ../ei/core/import_sequence_synonyms.pl ../ei/conf/core-import.ini", "language": "shell", "name": "Step 2.2: Create database and load sequence data" } ] } [/block] [...read more](doc:step-22-create-database-and-load-sequence-data) [block:code] { "codes": [ { "code": "cd ~/import\nperl ../ei/core/prepare_gff.pl ../ei/conf/core-import.ini", "language": "shell", "name": "Step 2.3: Prepare the gff file for import" } ] } [/block] [...read more](doc:step-23-prepare-the-gff-file-for-import) [block:code] { "codes": [ { "code": "cd ~/import\nperl ../ei/core/import_gene_models.pl ../ei/conf/core-import.ini", "language": "shell", "name": "Step 2.4: Import gff from prepared file" } ] } [/block] [...read more](doc:step-24-import-gff-from-prepared-file) [block:code] { "codes": [ { "code": "cd ~/import\nperl ../ei/core/verify_translations.pl ../ei/conf/core-import.ini", "language": "shell", "name": "(optional) Step 2.5: Verify import" } ] } [/block] [...read more](doc:step-25-verify-import) [block:code] { "codes": [ { "code": "cd ~/import\nperl ../ei/core/import_blastp.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini\nperl ../ei/core/import_repeatmasker.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini\nperl ../ei/core/import_interproscan.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini\nperl ../ei/core/import_cegma_busco.pl ../ei/conf/example.ini ../ei/conf/core-import-extra.ini", "language": "shell", "name": "(optional) Step 2.6: Import additional annotations" } ] } [/block] [...read more](doc:step-26-import-additional-annotations) [block:code] { "codes": [ { "code": "cd ~/import\nperl ../ei/core/export_sequences.pl ../ei/conf/core-import.ini\nperl ../ei/core/export_json.pl ../ei/conf/core-import.ini", "language": "shell", "name": "(optional) Step 2.7: Export files" } ] } [/block] [...read more](doc:step-27-export-files) [block:code] { "codes": [ { "code": "cd ~/import\nperl ../ei/core/index_database.pl ../ei/conf/core-import.ini", "language": "shell", "name": "(optional) Step 2.8: Generate search index" } ] } [/block] [...read more](doc:step-28-generate-search-index) [block:api-header] { "type": "basic", "title": "Stage 3 - Web site configuration" } [/block] edit ``setup.ini`` to add ``operophtera_brumata_v1_core_31_84_1`` to ``[DATA_SOURCE] SPECIES_DBS`` [block:code] { "codes": [ { "code": "cd ~/ei/em\n./update-ensembl-code.sh ../conf/setup.ini", "language": "shell", "name": "Step 3.1: Update Ensembl webcode" } ] } [/block] [...read more](doc:step-31-update-ensembl-webcode) [block:code] { "codes": [ { "code": "cd ~/ei/em\n./reload-ensembl-site.sh ../conf/setup.ini", "language": "shell", "name": "Step 3.2: Reload Ensembl website" } ] } [/block] [...read more](doc:step-32-reload-ensembl-website) [block:api-header] { "type": "basic", "title": "(optional) Stage 4 - Compara import" } [/block] Setting up a compara requires the results of a set of analyses in addition to a ``.ini`` file similar to ``compara-import.ini`` (which contains the configuration used for the [lepbase.org](http://lepbase.org) compara). Once you have the appropriate files listed in [Stage 4 requirements](doc:stage-4-requirements), you will be ready to start importing a a compara.