{"__v":0,"_id":"58c10f5a2c3c720f00768bbc","api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"body":"## Dockerised setup\n\neasy-import has been incorporated as a core component of [GenomeHubs](http://genomehubs.org), which provide a Dockerised solution to setting up a custom [Ensembl](http://ensembl.org) genome browser with a [SequenceServer](http://sequenceserver.com) BLAST server and [h5ai](https://larsjung.de/h5ai/) downloads server.\n\n[PeerJ Preprints 5:e2401v2](https://doi.org/10.7287/peerj.preprints.2401v2)\n[block:callout]\n{\n  \"type\": \"success\",\n  \"title\": \"Very quick start...\",\n  \"body\": \"Install docker then:\\n\\n```\\ncd\\ngit clone https://github.com/genomehubs/demo\\ndemo/import.sh\\n```\\n\\n(best run as UID 1000)\"\n}\n[/block]\n\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/2153ef5-GenomeHubs_schematic_overview.png\",\n        \"GenomeHubs schematic overview.png\",\n        743,\n        519,\n        \"#131313\"\n      ],\n      \"caption\": \"Overview of GenomeHubs containers\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Stage 0 - Docker setup\"\n}\n[/block]\nTo run easy-import as part of [GenomeHubs](http://genomehubs.org) requires [Docker](http://www.docker.com). The easy import scripts can also be run independently, if required, but use outside of Docker is no longer supported.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"sudo apt-get update\\nsudo apt-get -y install apt-transport-https ca-certificates\\nsudo apt-key adv \\\\\\n               --keyserver hkp://ha.pool.sks-keyservers.net:80 \\\\\\n               --recv-keys 58118E89F3A912897C070ADBF76221572C52609D\\necho \\\"deb https://apt.dockerproject.org/repo ubuntu-xenial main\\\" | sudo tee /etc/apt/sources.list.d/docker.list\\nsudo apt-get update\\n\\nsudo apt-cache policy docker-engine\\nsudo apt-get -y install linux-image-extra-$(uname -r) linux-image-extra-virtual\\n\\nsudo apt-get -y install docker-engine\\n\\nsudo groupadd docker\\nsudo usermod -aG docker $USER\",\n      \"language\": \"shell\",\n      \"name\": \"Install Docker\"\n    }\n  ]\n}\n[/block]\nFor more detailed instructions and troubleshooting, please see the official [Docker documentation](https://docs.docker.com).\n[block:api-header]\n{\n  \"title\": \"Stage 1 - Database setup\"\n}\n[/block]\neasy-import requires a local mySQL instance with a preloaded Ensembl database to use as a template for populating production tables. These can be set up using GenomeHubs containers:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run -d \\\\\\n           --name genomehubs-mysql \\\\\\n           -e MYSQL_ROOT_PASSWORD=rootuserpassword \\\\\\n           -e MYSQL_ROOT_HOST='172.17.0.0/255.255.0.0' \\\\\\n           mysql/mysql-server:5.5\",\n      \"language\": \"shell\",\n      \"name\": \"Setup mySQL\"\n    }\n  ]\n}\n[/block]\nFull details of mySQL container configuration are available on the [mySQL Docker hub](https://hub.docker.com/_/mysql). For the simplest possible setup in this example we set the root password in the `docker run` command, though this is not the most secure option.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run -d \\\\\\n           --name genomehubs-ensembl \\\\\\n           -v ~/demo/genomehubs-import/conf:/ensembl/conf \\\\\\n           -v ~/demo/genomehubs-import/logs:/ensembl/logs \\\\\\n           --link example-mysql:mysql/mysql-server \\\\\\n           genomehubs/easy-mirror:latest /ensembl/scripts/database.sh /ensembl/conf/database.ini\",\n      \"language\": \"shell\",\n      \"name\": \"Setup Ensembl database users and template database\"\n    }\n  ]\n}\n[/block]\nWhere `~/demo/genomehubs-import/conf` contains a database.ini file as described in [Files: database.ini](doc:databaseini)\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Stage 2 - Core import\"\n}\n[/block]\nUsing ``core-import.ini`` will install a new core database for the winter moth *Operophtera brumata*\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run --rm \\\\\\n           --name easy-import-operophtera_brumata_v1_core_32_85_1 \\\\\\n           --link genomehubs-mysql \\\\\\n           -v ~/demo/genomehubs-import/import/conf:/import/conf \\\\\\n           -v ~/demo/genomehubs-import/import/data:/import/data \\\\\\n           -v ~/demo/genomehubs-import/download/data:/import/download \\\\\\n           -v ~/demo/genomehubs-import/blast/data:/import/blast \\\\\\n           -e DATABASE=operophtera_brumata_v1_core_32_85_1 \\\\\\n           -e FLAGS=\\\"-s -p -g -e -j -i\\\" \\\\\\n           genomehubs/easy-import:latest\",\n      \"language\": \"shell\",\n      \"name\": \"Import new assembly, export files and index database\"\n    }\n  ]\n}\n[/block]\n\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run --rm \\\\\\n           --name easy-import-melitaea_cinxia_core_32_85_1 \\\\\\n           --link genomehubs-mysql \\\\\\n           -v ~/demo/genomehubs-import/import/conf:/import/conf \\\\\\n           -v ~/demo/genomehubs-import/import/data:/import/data \\\\\\n           -v ~/demo/genomehubs-import/download/data:/import/download \\\\\\n           -v ~/demo/genomehubs-import/blast/data:/import/blast \\\\\\n           -e DATABASE=melitaea_cinxia_core_32_85_1 \\\\\\n           -e FLAGS=\\\"-e -i -j\\\" \\\\\\n           genomehubs/easy-import:latest\",\n      \"language\": \"shell\",\n      \"name\": \"Export files and index mirrored database\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"(optional) Stage 3 - Functional annotation\"\n}\n[/block]\nEnsmebl provides support for adding a diverse set of additional annotations. We provide containers to run some common analyses to produce output files that can be loaded into an Ensembl database by scripts within the EasyMirror container.  Further datatypes may be added by using some of the many import scripts, available in the Ensembl [github repositories](https://github.com/ensembl).\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Stage 4 - Startup web services\"\n}\n[/block]\n\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run -d \\\\\\n           --name genomehubs-h5ai \\\\\\n           -v ~/demo/genomehubs-import/download/conf:/conf \\\\\\n           -v ~/demo/genomehubs-import/download/data:/var/www/demo \\\\\\n           -p 8082:8080 \\\\\\n           genomehubs/h5ai:latest\",\n      \"language\": \"shell\",\n      \"name\": \"h5ai downloads server\"\n    }\n  ]\n}\n[/block]\n\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run -d \\\\\\n           --name genomehubs-sequenceserver \\\\\\n           -v ~/demo/genomehubs-import/blast/conf:/conf \\\\\\n           -v ~/demo/genomehubs-import/blast/data:/dbs \\\\\\n           -p 8083:4567 \\\\\\n           genomehubs/sequenceserver:latest\",\n      \"language\": \"shell\",\n      \"name\": \"SequenceServer BLAST server\"\n    }\n  ]\n}\n[/block]\n\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run -d \\\\\\n           --name genomehubs-ensembl \\\\\\n           -v ~/demo/genomehubs-import/ensembl/gh-conf:/ensembl/conf \\\\\\n           -v ~/demo/genomehubs-import/ensembl/logs:/ensembl/logs \\\\\\n           --link genomehubs-mysql \\\\\\n           -p 8081:8080 \\\\\\n           genomehubs/easy-mirror:latest\",\n      \"language\": \"text\",\n      \"name\": \"Ensembl genome browser\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"title\": \"Stage 5 - visit site\"\n}\n[/block]\nYour newly created GenomeHubs Ensembl site will now be available at http://127.0.0.1:8081, with h5ai and SequenceServer on ports 8082 and 8083, respectively.","category":"58c10f5a2c3c720f00768b88","createdAt":"2016-05-13T08:42:18.691Z","excerpt":"The easiest way to start importing data into an Ensembl","githubsync":"","hidden":false,"isReference":false,"link_external":false,"link_url":"","next":{"pages":[],"description":""},"order":0,"parentDoc":null,"project":"5735936aafab441700723a50","slug":"quick-start-guide","sync_unique":"","title":"Quick start guide","type":"basic","updates":[],"user":"573592b84b0ab120000b7d44","version":"58c10f5a2c3c720f00768b87","childrenPages":[]}

Quick start guide

The easiest way to start importing data into an Ensembl

## Dockerised setup easy-import has been incorporated as a core component of [GenomeHubs](http://genomehubs.org), which provide a Dockerised solution to setting up a custom [Ensembl](http://ensembl.org) genome browser with a [SequenceServer](http://sequenceserver.com) BLAST server and [h5ai](https://larsjung.de/h5ai/) downloads server. [PeerJ Preprints 5:e2401v2](https://doi.org/10.7287/peerj.preprints.2401v2) [block:callout] { "type": "success", "title": "Very quick start...", "body": "Install docker then:\n\n```\ncd\ngit clone https://github.com/genomehubs/demo\ndemo/import.sh\n```\n\n(best run as UID 1000)" } [/block] [block:image] { "images": [ { "image": [ "https://files.readme.io/2153ef5-GenomeHubs_schematic_overview.png", "GenomeHubs schematic overview.png", 743, 519, "#131313" ], "caption": "Overview of GenomeHubs containers" } ] } [/block] [block:api-header] { "type": "basic", "title": "Stage 0 - Docker setup" } [/block] To run easy-import as part of [GenomeHubs](http://genomehubs.org) requires [Docker](http://www.docker.com). The easy import scripts can also be run independently, if required, but use outside of Docker is no longer supported. [block:code] { "codes": [ { "code": "sudo apt-get update\nsudo apt-get -y install apt-transport-https ca-certificates\nsudo apt-key adv \\\n --keyserver hkp://ha.pool.sks-keyservers.net:80 \\\n --recv-keys 58118E89F3A912897C070ADBF76221572C52609D\necho \"deb https://apt.dockerproject.org/repo ubuntu-xenial main\" | sudo tee /etc/apt/sources.list.d/docker.list\nsudo apt-get update\n\nsudo apt-cache policy docker-engine\nsudo apt-get -y install linux-image-extra-$(uname -r) linux-image-extra-virtual\n\nsudo apt-get -y install docker-engine\n\nsudo groupadd docker\nsudo usermod -aG docker $USER", "language": "shell", "name": "Install Docker" } ] } [/block] For more detailed instructions and troubleshooting, please see the official [Docker documentation](https://docs.docker.com). [block:api-header] { "title": "Stage 1 - Database setup" } [/block] easy-import requires a local mySQL instance with a preloaded Ensembl database to use as a template for populating production tables. These can be set up using GenomeHubs containers: [block:code] { "codes": [ { "code": "docker run -d \\\n --name genomehubs-mysql \\\n -e MYSQL_ROOT_PASSWORD=rootuserpassword \\\n -e MYSQL_ROOT_HOST='172.17.0.0/255.255.0.0' \\\n mysql/mysql-server:5.5", "language": "shell", "name": "Setup mySQL" } ] } [/block] Full details of mySQL container configuration are available on the [mySQL Docker hub](https://hub.docker.com/_/mysql). For the simplest possible setup in this example we set the root password in the `docker run` command, though this is not the most secure option. [block:code] { "codes": [ { "code": "docker run -d \\\n --name genomehubs-ensembl \\\n -v ~/demo/genomehubs-import/conf:/ensembl/conf \\\n -v ~/demo/genomehubs-import/logs:/ensembl/logs \\\n --link example-mysql:mysql/mysql-server \\\n genomehubs/easy-mirror:latest /ensembl/scripts/database.sh /ensembl/conf/database.ini", "language": "shell", "name": "Setup Ensembl database users and template database" } ] } [/block] Where `~/demo/genomehubs-import/conf` contains a database.ini file as described in [Files: database.ini](doc:databaseini) [block:api-header] { "type": "basic", "title": "Stage 2 - Core import" } [/block] Using ``core-import.ini`` will install a new core database for the winter moth *Operophtera brumata* [block:code] { "codes": [ { "code": "docker run --rm \\\n --name easy-import-operophtera_brumata_v1_core_32_85_1 \\\n --link genomehubs-mysql \\\n -v ~/demo/genomehubs-import/import/conf:/import/conf \\\n -v ~/demo/genomehubs-import/import/data:/import/data \\\n -v ~/demo/genomehubs-import/download/data:/import/download \\\n -v ~/demo/genomehubs-import/blast/data:/import/blast \\\n -e DATABASE=operophtera_brumata_v1_core_32_85_1 \\\n -e FLAGS=\"-s -p -g -e -j -i\" \\\n genomehubs/easy-import:latest", "language": "shell", "name": "Import new assembly, export files and index database" } ] } [/block] [block:code] { "codes": [ { "code": "docker run --rm \\\n --name easy-import-melitaea_cinxia_core_32_85_1 \\\n --link genomehubs-mysql \\\n -v ~/demo/genomehubs-import/import/conf:/import/conf \\\n -v ~/demo/genomehubs-import/import/data:/import/data \\\n -v ~/demo/genomehubs-import/download/data:/import/download \\\n -v ~/demo/genomehubs-import/blast/data:/import/blast \\\n -e DATABASE=melitaea_cinxia_core_32_85_1 \\\n -e FLAGS=\"-e -i -j\" \\\n genomehubs/easy-import:latest", "language": "shell", "name": "Export files and index mirrored database" } ] } [/block] [block:api-header] { "type": "basic", "title": "(optional) Stage 3 - Functional annotation" } [/block] Ensmebl provides support for adding a diverse set of additional annotations. We provide containers to run some common analyses to produce output files that can be loaded into an Ensembl database by scripts within the EasyMirror container. Further datatypes may be added by using some of the many import scripts, available in the Ensembl [github repositories](https://github.com/ensembl). [block:api-header] { "type": "basic", "title": "Stage 4 - Startup web services" } [/block] [block:code] { "codes": [ { "code": "docker run -d \\\n --name genomehubs-h5ai \\\n -v ~/demo/genomehubs-import/download/conf:/conf \\\n -v ~/demo/genomehubs-import/download/data:/var/www/demo \\\n -p 8082:8080 \\\n genomehubs/h5ai:latest", "language": "shell", "name": "h5ai downloads server" } ] } [/block] [block:code] { "codes": [ { "code": "docker run -d \\\n --name genomehubs-sequenceserver \\\n -v ~/demo/genomehubs-import/blast/conf:/conf \\\n -v ~/demo/genomehubs-import/blast/data:/dbs \\\n -p 8083:4567 \\\n genomehubs/sequenceserver:latest", "language": "shell", "name": "SequenceServer BLAST server" } ] } [/block] [block:code] { "codes": [ { "code": "docker run -d \\\n --name genomehubs-ensembl \\\n -v ~/demo/genomehubs-import/ensembl/gh-conf:/ensembl/conf \\\n -v ~/demo/genomehubs-import/ensembl/logs:/ensembl/logs \\\n --link genomehubs-mysql \\\n -p 8081:8080 \\\n genomehubs/easy-mirror:latest", "language": "text", "name": "Ensembl genome browser" } ] } [/block] [block:api-header] { "title": "Stage 5 - visit site" } [/block] Your newly created GenomeHubs Ensembl site will now be available at http://127.0.0.1:8081, with h5ai and SequenceServer on ports 8082 and 8083, respectively.