Author SHA1 Message Date
Leopere ebf0a87c98 Fixup 2019-04-28 15:31:26 -04:00
Leopere 027d3872df Updating main README.md to match release status 2019-04-28 15:31:10 -04:00
Leopere eb5427469f Merge branch 'master' into refactor 2019-04-27 18:39:12 -04:00
Leopere de0a21a43c Merge branch 'refactor' of github.com:lbryio/lbry-docker into refactor 2019-04-27 17:52:47 -04:00
Leopere 6ca8e4431a Docker Hub doesn't like self referential markdown urs 2019-04-27 17:52:39 -04:00
Ryan McGuire 2092949655 Updates lbrycrd config file in testnet and regtest modes. 2019-04-27 17:35:40 -04:00
Leopere 10d4edc77d Adding image tags to README's 2019-04-27 17:17:37 -04:00
Leopere 6501267d30 Add chainquery tags 2019-04-27 17:15:23 -04:00
Leopere e8f6fa30d9 Cleaned up documentation
Closes #49
2019-04-27 17:09:49 -04:00
Leopere 0e537ef3e0 Updated documentation 2019-04-27 17:00:34 -04:00
Ryan McGuire 7c6f0b58a1 Adds VERSION ARG to the lbrynet compiler and adds more README. 2019-04-27 17:00:19 -04:00
Leopere 4580d06102 Fixed CONFIG_PATH issue in testing RUN_MODEs 2019-04-27 15:43:01 -04:00
Mark Beamer Jr 58682e1fbe removed permission escalation 2019-04-27 15:42:49 -04:00
Leopere 93ea84bb81 More removals
Renamed to fit with new nomenclature

Removed directory and renamed/moved Dockerfile

Updated COPY path

Moved docker-compose.yml for Lbrynet
Cleaned up comments a bit

Important notice to users in new comment

Updated compose example for image instead of build

updated paths

Added a step for chainquery-bootstrap setup

Moved and updated lbrycrd's env_file

Created lbrynet env_file

Moved and updated chainquery's env_file

Moved and updated lbrynet's compose file

More Cleanup

Added TODO's for checkmount.sh

Renamed compiler

Renamed production container

Final rename

fixup

fixup

Updated README refs

Just readability improvements

Packing directories with the repo

Git won't track empty directories but if you take the path of running these containers locally from within this repo you should have this directory.

Updated image refs to final production locations
2019-04-27 14:37:55 -04:00
Ryan McGuire 44f8f86308 Merges lbrynet compilers into one Dockerfile with ARG for BASE_IMAGE 2019-04-27 14:37:42 -04:00
Leopere c3accd79bd Adjusting to new structure
Replaced build with image docker-compose.yml-prod-example

Adjusted paths in volumes

Removed a few more things

healthcheck.sh which can be added later when completed.
renamed the Dockerfile to Dockerfile-x86_64-production

Updated image tag and added RUN_MODE

Updated image and added RUN_MODE

updated tag

Updated tag
2019-04-27 14:37:40 -04:00
Ryan McGuire 70ae63d02a Adds ENTRYPOINT to lbrynet to require mounted home directory before execution. 2019-04-27 14:37:40 -04:00
Ryan McGuire 69c8468e8f Refactor lbrynet compiler container locations and startup mechanics 2019-04-27 14:37:40 -04:00
Leopere f910732155 Removed traefik labels and moved data dir
Moved data dir and removed traefik label

Moved readme for testnet compose

Adding RUN_MODEs regtest and testnet

Adding advance_blocks.sh into all lbrycrd containers

Adding advance cmd and moved Dockerfile to appropriate dir

Cleanup and moved Dockerfiles

Moved lbrycrd Dockerfiles to appropriate subdirectories and removed.

More moving and deletions

Adding catchall output

Updated default runmode in example

Removed incomplete container directory

This should not be in a production branch at this point now that we have stable containers published.

Removed incomplete container for now

Deleted compile directory and README.md

Cleaning up chainquery

Deleting placeholders

Updated start.sh

Removing superfluous .dockerignores & more

Adjusted start.sh source path

Updated ignores

Updated readme in prep for build tags and ship armhf-prod dir

start.sh's should be recycled when possible.

No sense in code duplication lets try and aim for unification.

Docker prefers WORKDIR over cd

docker prefers WORKDIR over cd
2019-04-27 14:37:38 -04:00
Leopere 832c2f6b1b Merge branch 'master' into lbrynet 2019-04-23 23:04:51 -04:00
Mark Beamer Jr 607f8839ea updated maintainer of this to be me. 2019-04-23 21:22:18 -04:00
Ryan McGuire 1d3e5b7a39 Three stage Dockerfile for lbrynet ARM and X86_64 2019-04-23 21:07:38 -04:00
Ryan McGuire f0b0c6da63 Adds multi-stage Dockerfile for chainquery.
This allows unmodified configs to be mounted.
Also updates the system-cloud-init HOWTO for chainquery.
2019-04-22 16:17:18 -04:00
Leopere 868c00b4cb Adding what I had in the lbryio/lbry PR 2019-04-22 20:10:07 +00:00
Leopere 846bc84f19 Merge branch 'master' into lbrynet 2019-04-21 18:46:46 +00:00
Ryan McGuire 860b5b4fa5 Adds cloud-init systemd HOWTO for lbrycrd on DigitalOcean. 2019-04-17 13:59:05 -04:00
Leopere aff973f5a9 Clarification on comments and TODO: cleanups.
Documented the RUN_MODEs and just did some re-organizing.
2019-04-17 10:38:56 -04:00
Leopere d82e79100f Merge pull request #47 from EnigmaCurry/setuid
Adds new multi-stage Dockerfile for lbrycrd and mountable config files.
2019-04-16 19:44:54 -04:00
Ryan McGuire 7b063e217f Adds new multi-stage Dockerfile for lbrycrd and mountable config files.
This Dockerfile was originally outlined by @Leopere here:
https://github.com/lbryio/lbry-docker/pull/46#issuecomment-483779947
2019-04-16 19:00:39 -04:00
Mark Beamer Jr 1d575c28cf upgraded regtest dockerfile to 0.12.4 of lbrycrd 2019-02-23 11:26:07 -05:00
Mark fbbf6d9eb1 Merge pull request #44 from lbryio/testnet
commit running testnet with docker
2019-02-05 23:47:04 -05:00
Leopere c1cfd936fa Readme tweaking for modernized container update 2018-11-29 20:01:58 +00:00
Leopere b285a49227 Container builds and runs now.
Entrypoint needs updated to use entrypoint but otherwise stable.
2018-11-29 19:37:32 +00:00
Leopere 992a2ab7c9 updated home dir 2018-11-29 19:23:34 +00:00
Leopere cedb74dc3d On second thought don't declare a VOLUME here. 2018-11-29 19:17:49 +00:00
Leopere 38320c2204 Add placeholder healthcheck.sh 2018-11-29 19:16:46 +00:00
Leopere 2db22bccd1 Merge branch 'master' into lbrynet 2018-11-29 19:14:18 +00:00
Leopere 020344c481 Network name needs to not match this users name 2018-11-29 19:11:36 +00:00
Leopere 083c9f3906 Updated networking to match master 2018-11-29 19:09:05 +00:00
Leopere 1886f87b06 Merge branch 'master' into lbrynet 2018-11-29 19:08:01 +00:00
Leopere d37a932774 Merge branch 'master' into lbrynet 2018-11-29 19:04:50 +00:00
Leopere 0d0c88c850 fixup 2018-11-29 19:02:45 +00:00
Leopere 79c08b66f2 Completed ToDo item. 2018-11-29 18:46:23 +00:00
Leopere b206c5f3b4 Clarification added to port 50001 2018-11-29 18:45:53 +00:00
Leopere c8ceb79651 Just expose the ports it doesn't really pose any risks. 2018-11-29 18:40:19 +00:00
Leopere 062c72c5f4 Updated Dockerfile, Entrypoint and start
Dockerfile, cleanup and added start.sh & docker-entrypoint.sh
chmod +x on lbrynet/stuff/*.sh
2018-11-29 18:29:19 +00:00
Leopere adebf46603 Add start.sh for lbrynet 2018-11-29 18:20:32 +00:00
Leopere 68824f95e7 Cleanup and a bit of modernization in invocation 2018-11-29 15:25:59 +00:00
Leopere 9a67d873f9 More WIP tags 2018-11-26 23:14:00 +00:00
Leopere aae9221c72 Added WIP tag to README header 2018-11-26 23:13:06 +00:00
Leopere 1d6e17c810 Adding stuff dir and start/entrypoint scripts 2018-11-26 23:11:50 +00:00
Leopere ada255021c Added todo items into lbrynet compose.yml 2018-11-26 23:09:22 +00:00
Leopere 32d7abf54e Adding todo list items for lbrynet Dockerfile 2018-11-26 23:07:07 +00:00
62 changed files with 782 additions and 975 deletions

No files matched your search

-2
View File
@@ -1,2 +0,0 @@
data/
*/data/
+11 -5
View File
@@ -1,7 +1,13 @@
.gitlab-ci.yml
.DS_Store
*/docker-compose.override.yml
*/*/data
./docker-compose.yml
*.zip
*/docker-compose.override.yml
*/*/data
*.zip
*.tar.gz
*.tar.xz
./persist/lbrycrd/*
./persist/lbrynet/*
./persist/chainquery/*
+49 -28
View File
@@ -1,63 +1,84 @@
# lbry-Docker
## Scope
This repository is in heavy flux as it travels towards [version 1.0](https://github.com/lbryio/lbry-docker/projects/1) however its goal is to make development for and adoption of any of the LBRY appliances trivial. You should be able to clone pull fork your way to a better LBRY without having to do much more than some light reading of a README to get started.
## Tags
<!-- Document tags and link to their Dockerfiles here. -->
#### Documentation is WIP
Currently this repository is a WIP and is under heavy construction, use at your own risk make sure you keep regular backups of your wallets. Your milage may vary as how far this will work for you be sure to file good and concise issues if you plan to and keep in mind we're allergic to regressions when filing PR's.
## Scope
This repository aims to track and maintain most of the Dockerfiles within the LBRY Inc ecosystem.
Currently it supports the following in various form factors:
* Lbrycrd
* Lbrynet
* Chainquery
If you see an architecture that you have a use case for that you don't see in this list and it's stopping you from innovating on the LBRY blockchain search and bump an existing request or create a feature request ASAP on the issue tracker and we will do our best to support your architecture or build requirements.
The goals here will be to create containers that can deploy across a number of technologies namely `docker-compose`, `docker` itself, and Kubernetes k8's.
There will be a few edge cases where this repository will support installing these containers as a service on a machine using `cloud-init` however those come as a YMMV(your mileage may vary) warranty void if used.
We may suggest the use of well commented **`Shell scripts`**, Dockerfiles, and Docker-Compose which is a template that is a baseline for many other container management services such as [RancherOS](https://rancher.com/rancher-os/) and [Kubernetes](https://kubernetes.io/).
#### Goals
This repository aims for [Docker Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) minimalism, ephemerality, and statelessness. It will use well commented **`Shell scripts`**, Dockerfiles, and Docker-Compose which is a template that is a baseline for many other container management services such as [RancherOS](https://rancher.com/rancher-os/) and [Kubernetes](https://kubernetes.io/).
We have a number of priorities within this docker container repository.
Loosely defined our priorities are in the order as follows ordered highest priority to lowest.
1. Congruency - Containers should all behave similarly.
2. Safety - Cautioning the user when they're in a danger zone when possible.
3. Reliability - Only pushing functional tested containers to production allowing users to impose an update schema.
4. Ephemerality/Statelessness - You should be able to define the entire state in a cross cloud platform way such that the container itself should contain no state and thus have no reason to rely on it's specific contents as best as possible.
5. Works by default - As best as possible we will aim for these containers to work by default out of the box then if you wish you can find ways to improve upon what we've set out for the user on your own.
6. Flexibility - Despite all of the above we want people to be able to utilize all of the functionalities of the appliances contained within these containers. This means if you find that our system/implementation gets in your way this is one of our priorities to fix this.
In summary, this repository aims for [Docker Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) minimalism, ephemerality, and statelessness.
#### Beautiful Screenshot/Gif
Since there is nothing to see here just yet I'll entertain you with the current state of affairs with this repository.
![image](https://spee.ch/855d1958650b850b249b9ee592ba2f4c6fc7eeec/container-unloading-gone-wrong-151175.gif)
Here's an example tutorial we have for how to get some of our appliances up and running in minutes on DigitalOcean all hosted on our very own platform https://spee.ch a link to the guide is [here](https://github.com/lbryio/lbry-docker/blob/master/contrib/systemd-cloud-init.md) it contains a few lines of information and some copy paste-able code!
[![Video of creating lbrycrd droplet on DigitalOcean](https://spee.ch/@EnigmaCurry:d/lbrycrd-video-thumb.jpg)](https://spee.ch/@EnigmaCurry:d/lbrycrd-docker-cloud-init.mp4)
## Installation
[This is currently WIP and Not Recommended for Production](https://github.com/lbryio/lbry-docker/projects/1)
See [Running from source](##Running-from-source) for the current instructions on how to use this.
There are a few methods we're planning on releasing over time however two notable options are our [cloud-init](https://github.com/lbryio/lbry-docker/blob/master/contrib/systemd-cloud-init.md) install of lbrycrd and optionally chainquery on DigitalOcean.
Alternatively there is the [shell installer](https://github.com/lbryio/lbry-docker/blob/master/chainquery/README.md) that can run on any Ubuntu 18.04-LTS equipped machine to get you up and running with a database seed which can save you loads of time bootstrapping your instance.
#### Currently supported platforms
**X64 cpu architecture**
**X86_64 CPU architecture**
Lbrynet production & compiler, Lbrycrd, Chainqery are currently supported.
**Some arm architectures**
Lbrynet armhf and arm64 compilers
**More will be added on request and over time**
## Usage
For now I don't recommend using this container cluster however you're welcomed to [contribute](#contributing) if you feel up to the task.
You may see above [the installation section above](https://github.com/lbryio/lbry-docker#installation) to obtain container specific instructions.
## Running from source
Running this stuff from source should be possible if you have both [Docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) both installed. If these are both installed you can proceed to run the following from within your development directory.
Running this stuff from source should be possible if you have both [Docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) both installed. If these are both installed, you can proceed to run the following from within your development directory.
```
git clone https://github.com/lbryio/lbry-docker.git
```
Once you have a local copy of the recent source you will want to consider what containers/applications you require in your environment. At the moment since at the writing of this documentation this comes with an assertion of [YMMV](https://dictionary.cambridge.org/dictionary/english/ymmv) so if something isn't working feel free to make suggestions in the form of a PR for how we should do this better. The beauty of Open Source is learning better ways to do things as well as contributing to the world so I'm always going to be welcoming to contributions.
Once you have a local copy of the recent source, you will want to consider what containers/applications you require in your environment. At the moment since at the writing of this documentation, this comes with an assertion of [YMMV](https://dictionary.cambridge.org/dictionary/english/ymmv) so if something isn't working feel free to make suggestions in the form of a PR for how we should do this better. The beauty of Open Source is learning better ways to do things as well as contributing to the world, so I'm always going to be welcoming to contributions.
#### From Source for Contributions
Running from source for contributing and Merge/Pull requests.
My goal is to make contributing to this possible using Docker and also GitLab CI/CD time.
## [Contributing](CONTRIBUTING.md)
Keep in mind [I am](https://github.com/leopere/) preferential to receiving patches over rule following as we can always nudge you in the right direction to get things more compatible with the project ethos if it's not. Never be afraid to file a PR no one should be offended. This said following the next two guides will greatly improve the speed at which we can integrate your improvements.
Keep in mind [I am](https://github.com/leopere/) preferential to receiving patches over rule-following as we can always nudge you in the right direction to get things more compatible with the project ethos if it's not. Never be afraid to file a PR no one should be offended. Having said this following the next two guides will greatly improve the speed at which we can integrate your improvements.
* [Repository Standards]( https://lbry.tech/resources/repository-standards)
* [Contribute](https://lbry.tech/contribute)
* Have a LBC wallet ready as we want you to have some for the help! Hell why not post it in your Commit or Merge Request for all I care but take your tips!
* Have an LBC wallet ready as we want you to have some for the help! Hell, why not post it in your Commit or Merge Request for all I care but take your tips!
## Getting Support
#### Debugpaste [WIP]
I'll be including a function to get a self destructing debugpaste of your LBRY appliances logs you'll be able to execute something similar to the following in all containers to export raw logs to a paste service where you can then either modify them removing sensitive data or just take that URL and create a new issue after you [(Use Issue Search)](https://github.com/lbryio/lbry-docker/issues?utf8=%E2%9C%93&q=is%3Aissue) to make sure there isn't already an open thread for your issue.
#### Example debugpaste
```
cd chainquery/
docker-compose exec chainquery debugpaste
https://haste.nixc.us/ocatumatozaq.nginx
```
You can then take output given in the response from the debugpaste command and put that into your github issue (This may be automated at some point to some degree).
Sometimes we just aren't omniscient omnipresent or even omnipotent. So since this is the case despite our constant striving for these goals we're aware that things are imperfect so when you find these imperfections please reach out to us! You can become a contributor without even knowing how to code a damn thing! Just open a Git Issue so long as it's not a duplicate its a huge help to us and often times it'll earn you some LBC if you're a part of our community.
#### Creating an Issue
Please be sure to fill out the [issue template](https://github.com/lbryio/lbry-docker/issues/new) as best as possible. This will help us answer your questions faster and fix things quicker if we have quality reports.
-5
View File
@@ -1,5 +0,0 @@
data/
data.z*
chainquery.z*
*.zip
compile/
-52
View File
@@ -1,52 +0,0 @@
## This base image is for running latest chainquery
FROM ubuntu:18.04 as prep
LABEL MAINTAINER="leopere [at] nixc [dot] us"
## Install everything needed to unzip the file containing Chainquery
RUN apt-get update && \
apt-get -y install unzip curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/* && \
mkdir /download
## Download and extract the latest Zip containing Linux binary for Chainquery.
RUN curl -L -o /download/chainquery.zip $(curl -s https://api.github.com/repos/lbryio/chainquery/releases | fgrep 'Linux_x86_64.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
cd /download && \
ls -lAh /download && pwd && \
unzip chainquery.zip && \
rm chainquery.zip
## TODO: Use this instead of everything above here at some point. To do this we will need the LBRY team to host all of their release binaries in their latest form internally I guess since github doesn't support latest tags.
# ADD --chown=1000:1000 https://github.com/lbryio/chainquery/releases/download/latest/chainquery_latest_Linux_x86_64.zip /download
# TODO: Switch to Alpine eventually we might need to compile in Alpine for this.
FROM ubuntu:18.04 as app
## Run as unprivileged user.
RUN adduser chainquery --gecos GECOS --shell /bin/bash --disabled-password --home /home/chainquery
# Pull chainquery executable from prep container.
COPY --from=0 /download/chainquery /usr/bin
# ADD --chown=1000:1000 https://github.com/lbryio/chainquery/releases/download/latest/chainquery_latest_Linux_x86_64.zip /usr/bin
# COPY stuff/chainqueryconfig.toml /etc/chainquery/
## Get latest chainqueryconfig.toml from master on github as template.
# TODO: add magic which pattern matches for chainqueryconfig.toml DEFAULT strings and replaces them with configured settings on container start.
ADD https://raw.githubusercontent.com/lbryio/chainquery/master/config/default/chainqueryconfig.toml /etc/chainquery/chainqueryconfig.toml.orig
## Install start.sh as executable script in container $PATH
COPY stuff/start.sh /usr/local/bin/start
# TODO: Implement docker-entrypoint if later required this might be handy if we need certain maintainence tasks to be executed in the live container.
# COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
# ## From here onward we're doing this with no root.
# USER chainquery
## Expose Chainquery API port
EXPOSE 6300
## Execute start script earlier installed into $PATH
CMD ["start"]
@@ -0,0 +1,23 @@
## Get the latest source and extract it for the app container.
## Design choices, two RUN layers intended to keep builds faster, the zipped
FROM ubuntu:18.04 as prep
LABEL MAINTAINER="leopere [at] nixc [dot] us"
RUN apt-get update && \
apt-get -y install unzip curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/*
WORKDIR /
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
COPY ./stuff/start.sh start
RUN curl -L -o /chainquery.zip $(curl -s https://api.github.com/repos/lbryio/chainquery/releases | grep -F 'Linux_x86_64.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
unzip /chainquery.zip && \
rm /chainquery.zip
FROM ubuntu:18.04 as app
ADD https://raw.githubusercontent.com/lbryio/chainquery/master/config/default/chainqueryconfig.toml /etc/lbry/chainqueryconfig.toml.orig
RUN adduser chainquery --gecos GECOS --shell /bin/bash --disabled-password --home /home/chainquery && \
chown -R chainquery:chainquery /etc/lbry
COPY --from=prep /chainquery /start /usr/bin/
EXPOSE 6300
USER chainquery
CMD ["start"]
+30 -21
View File
@@ -1,39 +1,46 @@
# Docker image tags
`lbry/chainquery`
`[linux-x86_64-production](https://github.com/lbryio/lbry-docker/blob/master/chainquery/Dockerfile-linux-x86_64-production)` (Latest release)
# Installation instructions for Chainquery and dependencies
## Preface
I have set up this tutorial so that you should be able to simply copy and paste the commands into your linux console. However if you run into issues please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail.
I have set up this tutorial so that you should be able to copy and paste the commands into your Linux console. However, if you run into issues, please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail.
## Caveats to installing Chainquery
Chainquery is an interface to an extensive index of data which has to parse an entire blockchain's history into a SQL database in a way that is easily query-able. The process of syncing your lbrycrd level.db files can take a long time depending on your deployment and then once your instance is synced you need to index the blockchain with Chainquery. All of this takes a fair bit of time or resources at first it is possible to deploy this stack without the shell scripts, and I will eventually include a docker-compose.yml file which gives you a baseline for deploying this to k8's and other platforms but for now single host deployment via shell script will be what's supported.
## Goals of this README.md
This readme is targeted towards people who need a copy pasta recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine you should be able to see what a running instance of Chainquery looks like and start writing apps against it's API calls etc.
This readme is targeted towards people who need a copy paste recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine, you should be able to see what a running instance of Chainquery looks like and start writing apps against its API.
## Dependencies
My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have will be `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine.
My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have are `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine.
### Docker
Docker
Docker is effectively the cross platform software package repository it allows you to ship an entire environment in what's referred to as a container. Implying that containers hold everything that is needed to ship what's inside effectively from one place to another with a degree of standard that makes it easy for everyone along the way to replicate what is needed for their step in the chain.
The other side of docker is it brings everything that we would normally have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handfull of easy to follow steps that should result in the same environment on everyones host.
Docker is effectively the cross-platform software package repository it allows you to ship an entire environment in what's referred to as a container. Containers are intended to hold everything that is needed to ship what's required to run an application from one place to another with a degree of a standard that makes it easy for everyone along the way to reproduce the environment for their step in the chain.
The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this will suffice. At the writing of this tutorial this was not the latest version of Docker.
The other side of docker is it brings everything that we would typically have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handful of easy to follow steps that should result in the same environment on everyone's host.
#### Just the link to installation instructions please
The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this is sufficient. At the writing of this tutorial, this was not the latest version of Docker.
#### Just the link to installation instructions, please
Instructions for installing on Ubuntu are at the link that follows:
https://docs.docker.com/install/linux/docker-ce/ubuntu/
If you plan on using other versions of OS you should at least aim to be a linux base with a x64 CPU and the appropriate minimum version of the dependencies.
If you plan on using other versions of OS you should at least aim to be a Linux base with an x86_64 CPU and the appropriate minimum version of the dependencies.
### Docker-compose
Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format but it's not a required prerequisite for getting this running you just need to have it installed successfully.
Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format, but it's not a required prerequisite for getting this running you need to have it installed successfully.
Install Docker Compose via this guide below, its important if you're using an older version of linux to use the official documentation from Docker.com because you will need the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer.
Install Docker Compose via this guide below, and it is essential if you're using an older version of Linux to use the official documentation from Docker.com because you require the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer.
#### Just the link to installation instructions please
#### Just the link to installation instructions, please
https://docs.docker.com/compose/install/
### Git
For now the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally fairly minimal.
For now, the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally reasonably minimal.
#### Just the instructions please
#### Just the instructions, please
`apt-get install git -y`
## Get the lbry-docker repository
@@ -43,14 +50,14 @@ For now the recommended install includes grabbing the latest git repository from
## Setup networking
You only need external networking if you plan on keeping your docker-compose files separate.
For the sake of modularity in the design of this git repository the plan is to give you examples to try and then you're supposed to move towards your own custom docker-compose configuration. We're going to create
a docker bridge network that is going to be managed externally to your usual docker-compose networks which are compose internal.
For the sake of modularity in the design of this git repository, the plan is to give you examples to try and then you're supposed to move towards your custom docker-compose configuration. We're going to create
a docker bridge network that is going to be managed externally to your usual docker-compose networks which are docker-compose internal.
`docker network create -d bridge --subnet=10.6.1.0/16 lbry-network`
## Make directories and set permissions
This is only required on host mounted volumes. (the default settings)
Setting permissions is only typically required on host mounted volumes. (the default settings)
`sudo mkdir -p ./lbry-docker/lbrycrd/data`
@@ -66,17 +73,19 @@ This is only required on host mounted volumes. (the default settings)
Wait for lbrycrd to reach the top of the blockchain (console output should noticeably slow down)
Once you've reached the top of the blockchain you can press `CTRL+C` to exit back to the linux shell.
Once you've reached the top of the blockchain, you can press `CTRL+C` to exit back to the Linux shell.
## Setup chainquery
Now that you're done syncing your own copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery and it's dependencies. Since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go.
Now you're finished syncing your copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery, and it's dependencies since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go.
1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using a LBRY.io provided database checkpoint snapshot. This should be basically copy paste-able series of commands to get your own staging instance up asap if you want Route number one, follow along with the rest of this README.md
2. Route number two: You can your own fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware.
1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using an LBRY.io provided database checkpoint snapshot. The checkpoint should be copy-paste-able series of commands to get your staging instance up asap if you want Route number one, follow along with the rest of this README.md
2. Route number two: You can your fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware.
`cd ../chainquery/`
`cat ./compose/docker-compose.yml-prod-example > docker-compose.yml`
`./quick-bootstrap.sh getdata`
`./quick-bootstrap.sh extract`
-4
View File
@@ -1,4 +0,0 @@
data/
data.z*
chainquery.z*
compile/
-16
View File
@@ -1,16 +0,0 @@
COMPOSE_PROJECT_NAME=chainquery
#########################
## Chainquery Settings ##
#########################
RPC_ALLOW_IP=10.5.1.3
DEBUGMODE=false
#################
## Mysql Creds ##
#################
MYSQL_SERVER=10.5.1.10
MYSQL_USER=changeme
MYSQL_PASSWORD=changeme
MYSQL_DATABASE=chainquery
MYSQL_ROOT_PASSWORD=changeme
-4
View File
@@ -1,4 +0,0 @@
data/
data.z*
chainquery.z*
compile/
-52
View File
@@ -1,52 +0,0 @@
## This base image is for running latest chainquery
FROM ubuntu:18.04 as prep
LABEL MAINTAINER="leopere [at] nixc [dot] us"
## Install everything needed to unzip the file containing Chainquery
RUN apt-get update && \
apt-get -y install unzip curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/* && \
mkdir /download
## Download and extract the latest Zip containing Linux binary for Chainquery.
RUN curl -L -o /download/chainquery.zip $(curl -s https://api.github.com/repos/lbryio/chainquery/releases | fgrep 'Linux_x86_64.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
cd /download && \
ls -lAh /download && pwd && \
unzip chainquery.zip && \
rm chainquery.zip
## TODO: Use this instead of everything above here at some point. To do this we will need the LBRY team to host all of their release binaries in their latest form internally I guess since github doesn't support latest tags.
# ADD --chown=1000:1000 https://github.com/lbryio/chainquery/releases/download/latest/chainquery_latest_Linux_x86_64.zip /download
# TODO: Switch to Alpine eventually we might need to compile in Alpine for this.
FROM ubuntu:18.04 as app
## Run as unprivileged user.
RUN adduser chainquery --gecos GECOS --shell /bin/bash --disabled-password --home /home/chainquery
# Pull chainquery executable from prep container.
COPY --from=0 /download/chainquery /usr/bin
# ADD --chown=1000:1000 https://github.com/lbryio/chainquery/releases/download/latest/chainquery_latest_Linux_x86_64.zip /usr/bin
# COPY stuff/chainqueryconfig.toml /etc/chainquery/
## Get latest chainqueryconfig.toml from master on github as template.
# TODO: add magic which pattern matches for chainqueryconfig.toml DEFAULT strings and replaces them with configured settings on container start.
ADD https://raw.githubusercontent.com/lbryio/chainquery/master/config/default/chainqueryconfig.toml /etc/chainquery/chainqueryconfig.toml.orig
## Install start.sh as executable script in container $PATH
COPY stuff/start.sh /usr/local/bin/start
# TODO: Implement docker-entrypoint if later required this might be handy if we need certain maintainence tasks to be executed in the live container.
# COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
# ## From here onward we're doing this with no root.
# USER chainquery
## Expose Chainquery API port
EXPOSE 6300
## Execute start script earlier installed into $PATH
CMD ["start"]
-57
View File
@@ -1,57 +0,0 @@
version: '3.4'
networks:
lbrynet:
external: true
services:
###########
## MYSQL ##
###########
## MariaDB is currently not supported and neither is later versions of MySQL this may change.
## https://hub.docker.com/r/_/mariadb/
mysql:
image: mysql:5.7.23
restart: always
networks:
lbrynet:
ipv4_address: 10.5.1.10
aliases:
- mysql
env_file:
- .env
expose:
- 3306
## TODO: I want to find a way that is acceptable to everyone to lock this up
## and not share it with everyone at least eventually.
ports:
- 3306:3306
volumes:
- ./data/db:/var/lib/mysql
- ./stuff/my.cnf:/etc/mysql/conf.d/chainquery-optimizations.cnf
################
## Chainquery ##
################
chainquery:
build:
context: .
target: app
restart: always
networks:
lbrynet:
ipv4_address: 10.5.1.3
env_file:
- .env
- ../lbrycrd/.env
labels:
- "traefik.expose=false"
expose:
- 6300
ports:
- 6300:6300
depends_on:
- mysql
## TODO: Uncomment this in a docker-compose.override.yml to allow for external configurations.
# volumes:
# - ./data/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml
-148
View File
@@ -1,148 +0,0 @@
#!/usr/bin/env bash
## TODO: Be Polite and ask for confirmation.
function QandA() {
read -r -p "Continue with $1 [y/N] " response
response=${response,,} # tolower
if [[ "$response" =~ ^(yes|y)$ ]]; then
echo "Continuing with this."
eval $1
else
echo "Skipping the $1."
fi
}
## Check your $PATH for required dependencies.
## Stop and complain for now, later automagically install them.
## TODO: Add dependency checker.
function test_for_deps() {
## Test for Command
if ! [ -x "$(command -v $1)" ]; then
echo "Error: $1 is not installed." >&2
echo "You must have $1 installed."
else
echo "Info: $1 is installed."
fi
}
## Declare Linux app dependencies to check for.
DEPENDENCIES=(
docker
docker-compose
)
## TODO: Check for docker and docker-compose
function check_deps() {
for i in "${!DEPENDENCIES[@]}"; do
echo ${DEPENDENCIES[$i]}"_KEY"
## Indirect references http://tldp.org/LDP/abs/html/ivr.html
eval TESTDEP=\$"${DEPENDENCIES[$i]}"
test_for_deps $TESTDEP
done
}
function get_checkpoint() {
## Get DB Checkpoint data.
echo Asked to get the latest checkpoint data, downloading latest checkpoint.
echo This data is fairly large so this saves you a few days of parsing the LBRY blockchain.
docker run -v $(pwd)/:/download --rm leopere/axel-docker http://chainquery-data.s3.amazonaws.com/chainquery-data.zip -o ./chainquery.zip
}
#################################
## The real action begins here ##
#################################
## TODO: Add ways to get into and out of a bind here.
case $1 in
getdata )
if [[ -f ./chainquery.zip ]]; then
echo "Found a copy of ./chainquery.zip already in your system."
echo "We recommend that you delete this data before proceeding and grab a fresh copy."
QandA "rm -f ./chainquery.zip"
get_checkpoint
else
get_checkpoint
fi
;;
extract )
## Unpack the data again if need be.
echo Asked to unpack chainquery.zip if downloaded.
if [[ -f ./chainquery.zip ]]; then
docker run -v $(pwd)/:/data --rm leopere/unzip-docker ./chainquery.zip
else
echo "Could not extractas chainquery.zip did not exist."
echo "Feel free to execute './quick-bootstrap.sh getdata' first next time."
fi
;;
cleanup )
## Remove any junk here.
echo Asked to clean up leftover chainquery.zip to save on disk space.
rm chainquery.zip
;;
reset )
## Give up on everything and try again.
## TODO: Make it very obvious with a nice little Y/N prompt that you're about to trash your settings and start over.
echo "Agressively Killing all chainquery and dependency containers."
echo "executing: docker-compose kill"
docker-compose kill
echo "Cleaning up stopped containers."
echo "executing: docker-compose rm -f"
docker-compose rm -f
rm -Rf ./data
rm -f ./chainquery.zip
;;
start )
## Unsupported start command to start containers.
## You can use this if you want to start this thing gracefully.
## Ideally you would not use this in production.
echo "Asked to start chainquery gracefully for you."
echo "executing: docker-compose up -d mysql"
docker-compose up -d mysql
echo "giving mysql some time to establish schema, crypto, users, permissions, and tables"
sleep 30
echo "Starting Chainquery"
echo "executing: docker-compose up -d chainquery"
docker-compose up -d chainquery
## TODO: verify chainquery instance is up and healthy, this requires a functional HEALTHCHECK
echo "This should have chainquery up and running, currently theres no checks in this function to verify this however."
echo "Do feel free to execute 'docker-compose ps' to verify if its running and not restarting or exited."
echo "Final Note: You should try to use the docker-compose commands in the tutorial at https://github.com/lbryio/lbry-docker/blob/master/chainquery/README.md"
;;
compress-latest-checkpoint-data )
## This is not intended for public use.
docker-compose stop chainquery
docker-compose stop mysql
sudo zip -r chainquery-data.zip data
docker-compose up -d mysql
sleep 30
docker-compose up -d chainquery
;;
upload-latest-checkpoint-data )
## This is not intended for public use.
aws s3 cp ./chainquery-data.zip s3://chainquery-data/chainquery-data.new
aws s3 rm s3://chainquery-data/chainquery-data.zip
aws s3 mv s3://chainquery-data/chainquery-data.new s3://chainquery-data/chainquery-data.zip
;;
* )
echo "=================================================="
echo "You look like you need usage examples let me help."
echo "=================================================="
echo "./quick-boostrap.sh {Parameter}"
echo "Example: ./quick-bootstrap.sh getdata # Downloads the latest Chainquery checkpoint data from a LBRYio official aws instance."
echo ""
echo ""
echo "=================================================="
echo "Usage example and available parameters"
echo "=================================================="
echo ""
echo "getdata # This function grabs the latest Chainquery checkpoint data."
echo "extract # Unpacks the chainquery data into the correct directory. ./data/"
echo "cleanup # Removes chainquery.zip"
echo "reset # Reset the state of these containers entirely, use if all else fails."
echo ""
echo ""
echo "=================================================="
echo "=================================================="
echo "Any other functions that are not documented here are not intended for public use."
echo " These functions are included in this repository to keep things in one place."
;;
esac
@@ -1,16 +0,0 @@
## TODO: Don't hardcode this stuff for production
#Debug mode outputs specific information to the console
debugmode=false
#LBRYcrd URL is required for chainquery to query the blockchain
lbrycrdurl="rpc://lbryrpc:changeme@10.5.1.2:9245"
#MySQL DSN is required for chainquery to store information.
mysqldsn="changeme:changeme@tcp(10.5.1.10:3306)/chainquery"
#API MySQL DSN is required for chainquery to expose a SQL query service
apimysqldsn="changeme:changeme@tcp(10.5.1.10:3306)/chainquery"
#The command that should be executed to trigger a self update of the software. For linux, for example, `<yourscript>.sh`
#DEFAULT-autoupdatecommand=[unset]
@@ -1 +0,0 @@
#!/usr/bin/env bash
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-19
View File
@@ -1,19 +0,0 @@
COMPOSE_PROJECT_NAME=chainquery
#########################
## Chainquery Settings ##
#########################
## TODO: Test to ensure these vars can be pulled from the lbrycrd .env
#RPC_USER=lbryrpc ## Not super necessery to change this.
#RPC_PASSWORD=changeme ## Please replace changeme.
RPC_ALLOW_IP=10.5.1.3 ## You're better off not changing this.
#################
## Mysql Creds ##
#################
MYSQL_SERVER=10.5.1.10 ## You're better off not changing this.
MYSQL_USER=changeme ## This could be changed.
MYSQL_PASSWORD=changeme ## This could be set to something random it sets this string for both Mysql's main user and Chainquery's MysqlDSN.
MYSQL_DATABASE=chainquery ## This can stay the same.
MYSQL_ROOT_PASSWORD=changeme ## Set this to something random and obnoxious we're not using it.
-3
View File
@@ -1,3 +0,0 @@
#!/usr/bin/env bash
## TODO: Implement this at some point it requires CURL in the container.
curl http://localhost:6300/api/status
-9
View File
@@ -1,9 +0,0 @@
# Default Homebrew MySQL server config
[mysqld]
# Only allow connections from localhost
innodb_log_file_size=5G
key_buffer_size=1G
innodb_flush_log_at_trx_commit = 0
innodb_autoinc_lock_mode=2
innodb_buffer_pool_size=1G
innodb_log_buffer_size=1G
-54
View File
@@ -1,54 +0,0 @@
#!/usr/bin/env bash
## Config setup
## Setup Values
DEBUGMODE=$(echo "debugmode=$DEBUGMODE")
LBRYCRDURL=$(echo "lbrycrdurl=\"rpc://$RPC_USER:$RPC_PASSWORD@10.5.1.2:9245\"")
MYSQLDSN=$(echo "mysqldsn=\"$MYSQL_USER:$MYSQL_PASSWORD@tcp($MYSQL_SERVER:3306)/$MYSQL_DATABASE\"")
APIMYSQLDSN=$(echo "apimysqldsn=\"$MYSQL_USER:$MYSQL_PASSWORD@tcp($MYSQL_SERVER:3306)/$MYSQL_DATABASE\"")
## Setup Defaults
DEBUGMODE_DEFAULT='#DEFAULT-debugmode=false'
LBRYCRDURL_DEFAULT='#DEFAULT-lbrycrdurl="rpc://lbry:lbry@localhost:9245"'
MYSQLDSN_DEFAULT='#DEFAULT-mysqldsn="lbry:lbry@tcp(localhost:3306)/chainquery"'
APIMYSQLDSN_DEFAULT='#DEFAULT-apimysqldsn="lbry:lbry@tcp(localhost:3306)/chainquery"'
## Add setup value variable name to this list to get processed on container start
CONFIG_SETTINGS=(
DEBUGMODE
LBRYCRDURL
MYSQLDSN
APIMYSQLDSN
)
function set_configs() {
## Set configs on container start if not already set.
for i in "${!CONFIG_SETTINGS[@]}"; do
## Indirect references http://tldp.org/LDP/abs/html/ivr.html
eval FROM_STRING=\$"${CONFIG_SETTINGS[$i]}_DEFAULT"
eval TO_STRING=\$${CONFIG_SETTINGS[$i]}
## TODO: Add a bit more magic to make sure that you're only configuring things if not set by config mounts.
sed -i "s~$FROM_STRING~"$TO_STRING"~g" /etc/chainquery/chainqueryconfig.toml
done
echo "Reading config for debugging."
cat /etc/chainquery/chainqueryconfig.toml
}
if [[ ! -f /etc/chainquery/chainqueryconfig.toml ]]; then
echo "[INFO]: Did not find chainqueryconfig.toml"
echo " Installing default and configuring with provided environment variables if any."
## Install fresh copy of config file.
echo "cp -v /etc/chainquery/chainqueryconfig.toml.orig /etc/chainquery/chainqueryconfig.toml"
cp -v /etc/chainquery/chainqueryconfig.toml.orig /etc/chainquery/chainqueryconfig.toml
chmod 755 /etc/chainquery/chainqueryconfig.toml
ls -lAh /etc/chainquery/
set_configs
else
echo "[INFO]: Found a copy of chainqueryconfig.toml in /etc/chainquery"
echo " Attempting to non destructively install any new environment configurations."
set_configs
fi
## For now keeping this simple. Potentially eventually add all command args as envvars for the Dockerfile or use safe way to add args via docker-compose.yml
su -c "chainquery serve -c "/etc/chainquery/"" chainquery
@@ -19,7 +19,7 @@ services:
aliases:
- mysql
env_file:
- env
- ../environment/chainquery.env
expose:
- 3306
## TODO: I want to find a way that is acceptable to everyone to lock this up
@@ -27,23 +27,22 @@ services:
ports:
- 3306:3306
volumes:
- ./data/db:/var/lib/mysql
- ./stuff/my.cnf:/etc/mysql/conf.d/chainquery-optimizations.cnf
- ../persist/chainquery/db:/var/lib/mysql
## This one may need to be tweaked based on where you run this docker-compose from.
- ../stuff/my.cnf:/etc/mysql/conf.d/chainquery-optimizations.cnf
################
## Chainquery ##
################
chainquery:
build:
context: .
target: app
image: lbryio/lbry-docker:linux-x86_64-production
restart: always
networks:
lbry-network:
ipv4_address: 10.6.1.3
env_file:
- env
- ../lbrycrd/env
- ../environment/lbrycrd.env
labels:
- "traefik.expose=false"
expose:
@@ -53,5 +52,5 @@ services:
depends_on:
- mysql
## TODO: Uncomment this in a docker-compose.override.yml to allow for external configurations.
# volumes:
# - ./data/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml
volumes:
- ../persist/chainquery/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml
-1
View File
@@ -1 +0,0 @@
#!/usr/bin/env bash
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
+8 -11
View File
@@ -29,26 +29,23 @@ function set_configs() {
eval FROM_STRING=\$"${CONFIG_SETTINGS[$i]}_DEFAULT"
eval TO_STRING=\$${CONFIG_SETTINGS[$i]}
## TODO: Add a bit more magic to make sure that you're only configuring things if not set by config mounts.
sed -i "s~$FROM_STRING~"$TO_STRING"~g" /etc/chainquery/chainqueryconfig.toml
sed -i "s~$FROM_STRING~"$TO_STRING"~g" /etc/lbry/chainqueryconfig.toml
done
echo "Reading config for debugging."
cat /etc/chainquery/chainqueryconfig.toml
cat /etc/lbry/chainqueryconfig.toml
}
if [[ ! -f /etc/chainquery/chainqueryconfig.toml ]]; then
if [[ ! -f /etc/lbry/chainqueryconfig.toml ]]; then
echo "[INFO]: Did not find chainqueryconfig.toml"
echo " Installing default and configuring with provided environment variables if any."
## Install fresh copy of config file.
echo "cp -v /etc/chainquery/chainqueryconfig.toml.orig /etc/chainquery/chainqueryconfig.toml"
cp -v /etc/chainquery/chainqueryconfig.toml.orig /etc/chainquery/chainqueryconfig.toml
chmod 755 /etc/chainquery/chainqueryconfig.toml
ls -lAh /etc/chainquery/
echo "cp -v /etc/lbry/chainqueryconfig.toml.orig /etc/lbry/chainqueryconfig.toml"
cp -v /etc/lbry/chainqueryconfig.toml.orig /etc/lbry/chainqueryconfig.toml
ls -lAh /etc/lbry/
set_configs
else
echo "[INFO]: Found a copy of chainqueryconfig.toml in /etc/chainquery"
echo " Attempting to non destructively install any new environment configurations."
set_configs
echo "[INFO]: Found a copy of chainqueryconfig.toml in /etc/lbry"
fi
## For now keeping this simple. Potentially eventually add all command args as envvars for the Dockerfile or use safe way to add args via docker-compose.yml
su -c "chainquery serve -c "/etc/chainquery/"" chainquery
chainquery serve --configpath "/etc/lbry/"
+321
View File
@@ -0,0 +1,321 @@
# LBRY cloud-init with systemd
Contributing Author: [EnigmaCurry](https://www.enigmacurry.com)
Last Update: April 18 2019
This tutorial meant to be easy instructions for running a lbrycrd and chainquery
service on DigitalOcean. It's pretty much just copy-and-paste.
This tutorial should also work on any host that supports
[cloud-init](https://cloud-init.io/), but I've not tested it anywhere except for
DigitalOcean.
If you wish to use docker-compose, there is an [alternative
configuration](https://github.com/lbryio/lbry-docker/tree/master/lbrycrd)
for that. This tutorial uses cloud-init and systemd to control docker.
## It's easy to run your full lbrycrd node
[![Video of creating lbrycrd droplet on DigitalOcean](https://spee.ch/@EnigmaCurry:d/lbrycrd-video-thumb.jpg)](https://spee.ch/@EnigmaCurry:d/lbrycrd-docker-cloud-init.mp4)
## Installation
* Login to your DigitalOcean account and create a new droplet.
* Choose Ubuntu 18.04. (This will likely NOT work on other versions without tweaks.)
* Select a Standard droplet with 8GB of memory ($40 per month in 2019.)
* Select whatever datacenter you want.
* Mark the checkbox called `User data`, and paste the following into the box:
```
#cloud-config
## DigitalOcean user-data for Ubuntu 18.04 droplet
## Installs docker
## Setup systemd service for lbrycrd
## (This config runs docker on vanilla Ubuntu,
## it uses systemd in place of docker-compose or kubernetes.)
write_files:
- path: "/etc/lbry/lbrycrd.conf"
content: |
datadir=/data
port=9246
rpcuser=lbry
rpcpassword=lbry
rpcport=9245
rpcallowip=172.17.0.0/16
regtest=0
server=1
txindex=1
daemon=0
listen=1
- path: "/etc/systemd/system/lbrycrd.service"
content: |
[Unit]
Description=lbrycrd docker container
After=snap.docker.dockerd.service
Requires=snap.docker.dockerd.service
[Service]
Environment=SERVICE=lbrycrd
Environment=IMAGE=lbry/lbrycrd:linux-x86-64-production
TimeoutStartSec=0
ExecStartPre=-/snap/bin/docker stop $SERVICE
ExecStartPre=-/snap/bin/docker rm -f $SERVICE
ExecStartPre=-/snap/bin/docker pull $IMAGE
ExecStart=/snap/bin/docker run \
--rm \
--name lbrycrd \
-p 9246:9246 \
-p 127.0.0.1:9245:9245 \
--mount type=volume,source=lbrycrd-data,target=/data \
--mount type=bind,source=/etc/lbry/lbrycrd.conf,target=/etc/lbry/lbrycrd.conf \
-e RUN_MODE=default \
$IMAGE
ExecStop=/snap/bin/docker stop $SERVICE
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
- path: "/etc/mysql/conf.d/chainquery.cnf"
content: |
# Put MySQL optimizations specific to chainquery here
- path: "/etc/systemd/system/mysql.service"
content: |
[Unit]
Description=mysql docker container
After=snap.docker.dockerd.service
Requires=snap.docker.dockerd.service
[Service]
Environment=SERVICE=mysql
Environment=IMAGE=mysql:5
TimeoutStartSec=0
ExecStartPre=-/snap/bin/docker stop $SERVICE
ExecStartPre=-/snap/bin/docker rm -f $SERVICE
ExecStartPre=-/snap/bin/docker pull $IMAGE
ExecStart=/snap/bin/docker run \
--rm \
--name mysql \
--mount type=volume,source=mysql-data,target=/var/lib/mysql \
--mount type=bind,source=/etc/mysql/conf.d/chainquery.cnf,target=/etc/mysql/conf.d/chainquery.cnf \
-e MYSQL_USER=chainquery \
-e MYSQL_PASSWORD=chainquery \
-e MYSQL_DATABASE=chainquery \
-e MYSQL_ROOT_PASSWORD=chainquery \
$IMAGE
ExecStop=/snap/bin/docker stop $SERVICE
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
- path: "/etc/lbry/chainqueryconfig.toml"
content: |
### Reference config: https://raw.githubusercontent.com/lbryio/chainquery/master/config/default/chainqueryconfig.toml
lbrycrdurl="rpc://lbry:lbry@lbrycrd:9245"
mysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
apimysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
- path: "/etc/systemd/system/chainquery.service"
content: |
[Unit]
Description=chainquery docker container
After=mysql.service
Requires=mysql.service
Requires=snap.docker.dockerd.service
[Service]
Environment=SERVICE=chainquery
Environment=IMAGE=lbry/chainquery:linux-x86-64-production
TimeoutStartSec=0
ExecStartPre=-/snap/bin/docker stop $SERVICE
ExecStartPre=-/snap/bin/docker rm -f $SERVICE
ExecStartPre=-/snap/bin/docker pull $IMAGE
ExecStart=/snap/bin/docker run \
--rm \
--name chainquery \
-p 127.0.0.1:6300:6300 \
--mount type=bind,source=/etc/lbry/chainqueryconfig.toml,target=/etc/lbry/chainqueryconfig.toml \
--link mysql:mysql \
--link lbrycrd:lbrycrd \
$IMAGE
ExecStop=/snap/bin/docker stop $SERVICE
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
- path: "/root/.bash_aliases"
content: |
alias lbrycrd-cli="docker run --rm -it --link lbrycrd:lbrycrd --mount type=bind,source=/etc/lbry/lbrycrd.conf,target=/etc/lbry/lbrycrd.conf \
lbry/lbrycrd:linux-x86-64-production lbrycrd-cli -conf=/etc/lbry/lbrycrd.conf -rpcconnect=lbrycrd"
alias mysql="docker run --rm -it --link mysql:mysql mysql:5 mysql -hmysql -u chainquery --password=chainquery"
runcmd:
- apt-get update
- DEBIAN_FRONTEND=noninteractive apt-get -y upgrade
- snap install docker
- until /snap/bin/docker ps; do echo "Waiting for docker startup..."; sleep 1; done; echo "Docker is up."
- /snap/bin/docker volume create lbrycrd-data
- /snap/bin/docker volume create mysql-data
- systemctl enable --now lbrycrd
- echo "Good to go."
```
* You can leave everything above as it is, to use the default configuration, OR
you may edit the config in the box to your liking.
* For instance, if you wanted to run in [regtest
mode](https://lbry.tech/resources/regtest-setup), you would set
`regtest=1` in the first section under `write_files`.
* You can also edit the config files at any later point in `/etc/lbry`,
after you create the droplet.
* Select your SSH key so you can log in.
* Give it a good hostname.
* Click Create.
## Usage
### How to administer the system
Copy the IP address from the droplet status page, SSH into the droplet as root
using the same SSH key, you configured for the droplet.
The config file is in `/etc/lbry/lbrycrd.conf` on the host.
The systemd service is called `lbrycrd`, in
`/etc/systemd/system/lbrycrd.service`. It is preconfigured to start on system
startup.
#### Monitor the installer log
You can tail the log to monitor the install progress:
```
tail -f /var/log/cloud-init-output.log
```
Wait for the final `Good to go` message to know that the installer has finished.
(Press Ctrl-C to exit from tail.)
#### Check the status of the systemd service
You can interact with systemd using `systemctl` (status, start, stop, restart,
and more.) and `journalctl` (logging) tools.
```
systemctl status lbrycrd
```
```
journalctl --unit lbrycrd
```
[Here is a tutorial to get you familiarized with
systemd](https://www.digitalocean.com/community/tutorials/systemd-essentials-working-with-services-units-and-the-journal)
#### Check the container
You can get the same information directly from docker:
```
docker ps
```
```
docker logs lbrycrd
```
### Utilize lbrycrd-cli
You can use lbrycrd-cli from the host console. A bash alias gets added to
`/root/.bash_aliases` that invokes lbrycrd-cli in its own container.
```
$ lbrycrd-cli getinfo
{
"version": 120400,
"protocolversion": 70013,
"walletversion": 60000,
"balance": 0.00000000,
"blocks": 551965,
"timeoffset": 0,
"connections": 12,
"proxy": "",
"difficulty": 739465688254.7942,
"testnet": false,
"keypoololdest": 1555360604,
"keypoolsize": 101,
"paytxfee": 0.00000000,
"relayfee": 0.00001000,
"errors": ""
}
```
### Chainquery (optional)
The chainquery service is pre-installed but not enabled by default.
#### Enable and start the MySQL service
```
systemctl enable --now mysql
```
In case you need it, there is a bash alias called `mysql`.
(`/root/.bash_aliases`) for the MySQL client that allows you to log in to the
chainquery database.
#### Enable and start the chainquery service
The chainquery config file located on the host at: `/etc/lbry/chainqueryconfig.toml`
```
systemctl enable --now chainquery
```
In systemd, when you enable a service, it means always to start the service at
system boot. (`--now` means you also want to start the service right away.)
As with any service, you can control chainquery with `systemctl` and get logs
with `journalctl`:
##### Starting and stopping chainquery service
```
systemctl start chainquery
```
```
systemctl stop chainquery
```
##### Getting the chainquery service logs
```
journalctl --unit chainquery
```
(optionally use `-f` if you want to tail/follow the logs)
##### Disabling chainquery service
```
systemctl disable --now chainquery
```
### Known issues
Ubuntu's snap update mechanism will apparently [restart docker even if there are
no updates
available](https://github.com/lbryio/lbry-docker/pull/50#issuecomment-485435736).
In the future, this tutorial may replace the snap version of docker with the
regular PPA version of docker-ce, which has a more predictable update strategy
(apt-get) rather than auto-updates. More long term testing is needed to know
which way is better.
@@ -1,4 +1,4 @@
COMPOSE_PROJECT_NAME=chainquery
## Contains environment variables for chainquery containers
#########################
## Chainquery Settings ##
+1 -2
View File
@@ -1,5 +1,4 @@
COMPOSE_PROJECT_NAME=lbrycrd
## Contains environment variables for lbrycrd containers
#############
## Lbrycrd ##
#############
+2
View File
@@ -0,0 +1,2 @@
## Contains environment variables for lbrynet containers
RUN_MODE=default
-1
View File
@@ -1 +0,0 @@
data/
-45
View File
@@ -1,45 +0,0 @@
## This base image is for running latest lbrycrdd
# For some reason I may switch this image over to Alpine when I can RCA why it won't start.
FROM ubuntu:18.04
LABEL MAINTAINER="leopere [at] nixc [dot] us"
RUN addgroup --gid 1000 lbrycrd && \
adduser lbrycrd --uid 1000 --gid 1000 --gecos GECOS --shell /bin/bash --disabled-password --home /data && \
apt-get update && \
apt-get -y install unzip wget curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/*
## TODO: Consider adding debugpaste or variant
# RUN wget -O /usr/bin/debugpaste https://github.com/nixc-us/debugpaste-it/raw/master/bin/debugpaste_64 && \
# chmod +x /usr/bin/debugpaste
RUN wget -O /usr/bin/lbrycrd-linux.zip https://github.com/lbryio/lbrycrd/releases/download/v0.12.2.2/lbrycrd-linux.zip && \
cd /usr/bin/ && \
unzip lbrycrd-linux.zip && \
rm lbrycrd-linux.zip && \
chmod +x lbrycrdd lbrycrd-cli lbrycrd-tx
COPY stuff/debugpaste-it.sh /usr/local/bin/debugpaste-it
COPY stuff/start.sh /usr/local/bin/start
COPY stuff/docker-entrypoint.sh /usr/local/bin/docker-entrypoint
COPY stuff/healthcheck.sh /usr/local/bin/healthcheck
# USER lbrycrd
# RUN mkdir /data
VOLUME ["/data"]
WORKDIR /data
## TODO: Implement healthcheck.
# HEALTHCHECK ["healthcheck"]
## Exposing daemon port and RPC port
EXPOSE 9245 9246
## TODO: Decide what's important for lbrycrd and possibly add an entrypoint.
## Maybe catch things that might match things that can be easily executed in the
## lbrycrd cli and if nothing is entered just default to the containers shell.
## For now this is a placeholder that executes /bin/bash on `docker exec`
# ENTRYPOINT ["docker-entrypoint"]
CMD ["start"]
@@ -0,0 +1,33 @@
FROM ubuntu:18.04 as prep
LABEL MAINTAINER="leopere [at] nixc [dot] us"
## TODO: Implement version pinning. `apt-get install curl=<version>`
RUN apt-get update && \
apt-get -y install unzip curl build-essential && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/*
WORKDIR /
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
COPY ./stuff/start.sh start
COPY ./stuff/healthcheck.sh healthcheck
COPY ./stuff/advance_blocks.sh advance
COPY ./stuff/fix-permissions.c fix-permissions.c
RUN curl -L -o ./lbrycrd-linux.zip $(curl -s https://api.github.com/repos/lbryio/lbrycrd/releases | grep -F 'lbrycrd-linux.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
unzip ./lbrycrd-linux.zip && \
gcc fix-permissions.c -o fix-permissions && \
chmod +x ./lbrycrdd ./lbrycrd-cli ./lbrycrd-tx ./start ./healthcheck ./fix-permissions ./advance
FROM ubuntu:18.04 as app
COPY --from=prep /lbrycrdd /lbrycrd-cli /lbrycrd-tx /start /healthcheck /fix-permissions /advance /usr/bin/
RUN addgroup --gid 1000 lbrycrd && \
adduser lbrycrd --uid 1000 --gid 1000 --gecos GECOS --shell /bin/bash --disabled-password --home /data && \
mkdir /etc/lbry && \
chown lbrycrd /etc/lbry && \
chmod a+s /usr/bin/fix-permissions
VOLUME ["/data"]
WORKDIR /data
## TODO: Implement healthcheck.
# HEALTHCHECK ["healthcheck"]
EXPOSE 9246 9245
USER lbrycrd
CMD ["start"]
+4
View File
@@ -0,0 +1,4 @@
# lbrycrd
# Docker image tags
`lbry/lbrycrd`
`[linux-x86_64-production](https://github.com/lbryio/lbry-docker/blob/master/lbrycrd/Dockerfile-linux-x86_64-production)` (Latest release)
-1
View File
@@ -1 +0,0 @@
data/
-67
View File
@@ -1,67 +0,0 @@
###########################
## Compiler/Builder Step ##
###########################
FROM ubuntu:18.04 as builder
## TODO: Decide who gets the MAINTAINER label
## LABEL MAINTAINER="@Brannon Update Me Or Put Mine Here"
## Added apt work as single container layer then remove cruft before layer is committed.
RUN apt-get update && \
apt-get install -y build-essential libtool autotools-dev automake pkg-config bsdmainutils python3 git wget apt-utils && \
apt-get install -y libssl-dev libevent-dev libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libboost-locale-dev libboost-program-options-dev && \
apt-get install -y python-dev libbz2-dev autoconf ca-certificates clang-format && \
apt-get install -y software-properties-common && add-apt-repository -y ppa:bitcoin/bitcoin && apt-get update && apt-get install -y libdb4.8-dev libdb4.8++-dev && \
apt-get -y install unzip wget && \ ## Cleanup Post apt package installation.
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/* ## Others neednt play here(remove apt lists as no longer needed at this stage).
## Add debugpaste here for reporting compiler errors. Psuedo code.
## RUN wget -O /usr/local/bin/debugpaste https://url && \
## sha256sum check of debugpaste
########################
## Compile Steps Here ##
########################
## Compile and Test steps should likely be separate run commands/layers.
RUN ## Compile/Test steps here
####################
## App build step ##
####################
FROM ubuntu:18.04 as app
## Change this to pull from a certifiable source with supply chain in plain sight.
# RUN wget -O /usr/bin/debugpaste https://github.com/nixc-us/debugpaste-it/raw/master/bin/debugpaste_64 && \
# chmod +x /usr/bin/debugpaste
RUN adduser lbrycrd --gecos GECOS --shell /bin/bash --disabled-password --home /app
# ## This section can be replaced with a COPY to pull artifacts from build step.
# # RUN wget -O /usr/bin/lbrycrd-linux.zip https://github.com/lbryio/lbrycrd/releases/download/v0.12.2.2/lbrycrd-linux.zip && \
# # cd /usr/bin/ && \
# # unzip lbrycrd-linux.zip && \
# # rm lbrycrd-linux.zip && \
# # chmod +x lbrycrdd lbrycrd-cli lbrycrd-tx
# COPY --chown=1000:1000 /path/to/compiled/lbrycrd/bins /app/
## This needn't be included until debugpaste is ready.
# COPY debugpaste-it.sh /usr/local/bin/debugpaste-it
# COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
COPY start.sh /usr/local/bin/start
USER lbrycrdd
RUN mkdir -p /data/
VOLUME /data/
VOLUME /etc/lbrycrdd/
## Exposing daemon port and RPC port
EXPOSE 9245 9246
# ## For now this is a placeholder that executes /bin/bash on `docker exec`
# ENTRYPOINT ["docker-entrypoint"]
CMD ["start"]
View File
Whitespace-only changes.
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-56
View File
@@ -1,56 +0,0 @@
#!/usr/bin/env bash
# ## ToDo:
# ## Get a test case to see if this is the first run or a repeat run
# ## If it's a first run you need to do a full index including all transactions
# ## tx index creates an index of every single transaction in the block history if
# ## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
# ## This is specific to chainquery.
## Ensure perms are correct prior to running main binary
mkdir -p /data/.lbrycrd
chown -R 1000:1000 /data
chmod -R 755 /data
# chown -R 1000:1000 /etc/lbrycrdd
# chmod -R 755 /etc/lbrycrdd
rm -f /var/run/lbrycrdd.pid
## Set config params
echo 'rpcuser=lbryrpc\nrpcpassword='${RPC_PASSWORD:-changeme} > ~/.lbrycrd/lbrycrd.conf
echo "rpcallowip=${RPC_ALLOW_IP:-10.6.1.3}" >> ~/.lbrycrd/lbrycrd.conf
echo "rpcuser=${RPC_USER:-lbryrpc}" >> ~/.lbrycrd/lbrycrd.conf
## Control this invocation through envvar.
case ${RUN_MODE:-default} in
default )
lbrycrdd \
-server \
-conf=$HOME/.lbrycrd/lbrycrd.conf \
-printtoconsole
;;
reindex )
lbrycrdd \
-server \
-txindex \
-reindex \
-conf=$HOME/.lbrycrd/lbrycrd.conf \
-printtoconsole
;;
chainquery )
lbrycrdd \
-server \
-txindex \
-conf=$HOME/.lbrycrd/lbrycrd.conf \
-printtoconsole
;;
esac
## We were unsure if these function as intended so they were disabled for the time being.
# -port=${PORT:-9246} \
# -data=${DATA_DIR:-/data/} \
# -pid=${PID_FILE:/var/run/lbrycrdd.pid} \
# -rpcport=${RPC_PORT:-9245} \
# -rpcpassword=${RPC_PASSWORD:-changeme} \
# -rpcuser=${RPC_USER:-lbryrpc} \
# -rpcallowip=${RPC_ALLOW_IP:-10.6.1.3}
@@ -14,10 +14,8 @@ services:
networks:
lbry-network:
ipv4_address: 10.6.1.2
labels:
- "traefik.expose=false"
environment:
RUN_MODE: chainquery
RUN_MODE: default
env_file:
- env
expose:
@@ -25,4 +23,4 @@ services:
- 9246
## host volumes for persistent data such as wallet private keys.
volumes:
- "./data:/data"
- "../persist/data:/data"
@@ -5,13 +5,13 @@ services:
## Lbrycrd ##
#############
lbrycrd:
image: tiger5226/regtest
image: lbry/lbrycrd:linux-x86_64-production
restart: always
labels:
- "traefik.expose=true"
ports:
- "11336:9246"
- "11337:11337"
## host volumes for persistent data such as wallet private keys.
volumes:
- "./data:/data"
- "../persist/data:/data"
environment:
- RUN_MODE=regtest
@@ -5,13 +5,13 @@ services:
## Lbrycrd ##
#############
lbrycrd:
image: tiger5226/testnet
image: lbry/lbrycrd:linux-x86_64-production
restart: always
labels:
- "traefik.expose=true"
ports:
- "11336:9246"
- "11337:11337"
## host volumes for persistent data such as wallet private keys.
volumes:
- "./data:/data"
- "../persist/data:/data"
environment:
- RUN_MODE=testnet
-41
View File
@@ -1,41 +0,0 @@
## This base image is for running latest lbrycrdd
# For some reason I may switch this image over to Alpine when I can RCA why it won't start.
FROM ubuntu:18.04
LABEL MAINTAINER="leopere [at] nixc [dot] us"
RUN addgroup --gid 1000 lbrycrd && \
adduser lbrycrd --uid 1000 --gid 1000 --gecos GECOS --shell /bin/bash --disabled-password --home /data && \
apt-get update && \
apt-get -y install unzip wget curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/*
## TODO: Consider adding debugpaste or variant
# RUN wget -O /usr/bin/debugpaste https://github.com/nixc-us/debugpaste-it/raw/master/bin/debugpaste_64 && \
# chmod +x /usr/bin/debugpaste
RUN wget -O /usr/bin/lbrycrd-linux.zip https://github.com/lbryio/lbrycrd/releases/download/v0.12.3.1/lbrycrd-linux.zip && \
cd /usr/bin/ && \
unzip lbrycrd-linux.zip && \
rm lbrycrd-linux.zip && \
chmod +x lbrycrdd lbrycrd-cli lbrycrd-tx
COPY start.sh /usr/local/bin/start
COPY advance_blocks.sh /usr/local/bin/advance
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
# USER lbrycrd
# RUN mkdir /data
VOLUME ["/data"]
WORKDIR /data
## Exposing daemon port and RPC port
EXPOSE 9245 9246
## TODO: Decide what's important for lbrycrd and possibly add an entrypoint.
## Maybe catch things that might match things that can be easily executed in the
## lbrycrd cli and if nothing is entered just default to the containers shell.
## For now this is a placeholder that executes /bin/bash on `docker exec`
# ENTRYPOINT ["docker-entrypoint"]
CMD ["start"]
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-34
View File
@@ -1,34 +0,0 @@
#!/usr/bin/env bash
# ## ToDo: Get a test case to see if this is the first run or a repeat run.
# ## If it's a first run you need to do a full index including all transactions
# ## tx index creates an index of every single transaction in the block history if
# ## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
# ## This is specific to chainquery.
## Ensure perms are correct prior to running main binary
mkdir -p /data/.lbrycrd
chown -R lbrycrd:lbrycrd /data
chmod -R 755 /data/
## TODO: Consider a config directory for future magic.
# chown -R 1000:1000 /etc/lbrycrd
# chmod -R 755 /etc/lbrycrd
rm -f /var/run/lbrycrd.pid
## Set config params
## TODO: Make this more automagic in the future.
echo "rpcuser=lbry" > /data/.lbrycrd/lbrycrd.conf
echo "rpcpassword=lbry" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcport=11337" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcbind=0.0.0.0" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcallowip=0.0.0.0/0" >> /data/.lbrycrd/lbrycrd.conf
echo "regtest=1" >> /data/.lbrycrd/lbrycrd.conf
echo "txindex=1" >> /data/.lbrycrd/lbrycrd.conf
echo "server=1" >> /data/.lbrycrd/lbrycrd.conf
echo "printtoconsole=1" >> /data/.lbrycrd/lbrycrd.conf
#nohup advance &>/dev/null &
su -c "lbrycrdd -conf=/data/.lbrycrd/lbrycrd.conf" lbrycrd
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
docker build -t tiger5226/regtest:latest .
docker push tiger5226/regtest:latest
@@ -2,4 +2,4 @@
while true; do
lbrycrd-cli -conf=/data/.lbrycrd/lbrycrd.conf generate 1 >> /tmp/output.log
sleep 2
done
done
-3
View File
@@ -1,3 +0,0 @@
#!/usr/bin/env bash
echo "placeholder for now, come back soon for more debugging magic"
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
+9
View File
@@ -0,0 +1,9 @@
#include <unistd.h>
int main() {
// This program needs to run with setuid == root
// This needs to be in a compiled language because you cannot setuid bash scripts
setuid(0);
execle("/bin/bash", "bash", "-c",
"/bin/chown -R lbrycrd:lbrycrd /data && /bin/chmod -R 755 /data/",
(char*) NULL, (char*) NULL);
}
+71 -27
View File
@@ -1,40 +1,84 @@
#!/usr/bin/env bash
CONFIG_PATH=/etc/lbry/lbrycrd.conf
function set_config() {
CONFIG_PATH=/etc/lbry/lbrycrd.conf
if [ -f "$CONFIG_PATH" ]
then
echo "Using the config file that was mounted into the container."
else
echo "Creating a fresh config file from environment variables."
## Set config params
echo "rpcuser=$RPC_USER" > $CONFIG_PATH
echo "rpcpassword=$RPC_PASSWORD" >> $CONFIG_PATH
echo "rpcallowip=$RPC_ALLOW_IP" >> $CONFIG_PATH
echo "rpcport=9245" >> $CONFIG_PATH
echo "rpcbind=0.0.0.0" >> $CONFIG_PATH
#echo "bind=0.0.0.0" >> $CONFIG_PATH
fi
}
# ## ToDo: Get a test case to see if this is the first run or a repeat run.
# ## If it's a first run you need to do a full index including all transactions
# ## tx index creates an index of every single transaction in the block history if
# ## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
# ## This is specific to chainquery.
## Ensure perms are correct prior to running main binary
mkdir -p /data/.lbrycrd
chown -R lbrycrd:lbrycrd /data
chmod -R 755 /data/
/usr/bin/fix-permissions
## TODO: Consider a config directory for future magic.
# chown -R 1000:1000 /etc/lbrycrd
# chmod -R 755 /etc/lbrycrd
rm -f /var/run/lbrycrd.pid
## Set config params
## TODO: Make this more automagic in the future.
echo "rpcuser=$RPC_USER" > /data/.lbrycrd/lbrycrd.conf
echo "rpcpassword=$RPC_PASSWORD" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcallowip=$RPC_ALLOW_IP" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcport=9245" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcbind=0.0.0.0" >> /data/.lbrycrd/lbrycrd.conf
#echo "bind=0.0.0.0" >> /data/.lbrycrd/lbrycrd.conf
## Control this invocation through envvar.
## You can optionally specify a run mode if you want to use lbry defined presets for compatibility.
case $RUN_MODE in
default )
su -c "lbrycrdd -server -conf=/data/.lbrycrd/lbrycrd.conf -printtoconsole" lbrycrd
set_config
lbrycrdd -server -conf=$CONFIG_PATH -printtoconsole
;;
## If it's a first run you need to do a full index including all transactions
## tx index creates an index of every single transaction in the block history if
## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
## This is generally specific to chainquery.
reindex )
su -c "lbrycrdd -server -txindex -reindex -conf=/data/.lbrycrd/lbrycrd.conf -printtoconsole" lbrycrd
## Apply this RUN_MODE in the case you need to update a dataset. NOTE: you do not need to use `RUN_MODE reindex` for more than one complete run.
set_config
lbrycrdd -server -txindex -reindex -conf=$CONFIG_PATH -printtoconsole
;;
chainquery )
su -c "lbrycrdd -server -txindex -conf=/data/.lbrycrd/lbrycrd.conf -printtoconsole" lbrycrd
## If your only goal is to run Chainquery against this instance of lbrycrd and you're starting a
## fresh local dataset use this run mode.
set_config
lbrycrdd -server -txindex -conf=$CONFIG_PATH -printtoconsole
;;
regtest )
## Set config params
## TODO: Make this more automagic in the future.
mkdir -p `dirname $CONFIG_PATH`
echo "rpcuser=lbry" > $CONFIG_PATH
echo "rpcpassword=lbry" >> $CONFIG_PATH
echo "rpcport=11337" >> $CONFIG_PATH
echo "rpcbind=0.0.0.0" >> $CONFIG_PATH
echo "rpcallowip=0.0.0.0/0" >> $CONFIG_PATH
echo "regtest=1" >> $CONFIG_PATH
echo "txindex=1" >> $CONFIG_PATH
echo "server=1" >> $CONFIG_PATH
echo "printtoconsole=1" >> $CONFIG_PATH
#nohup advance &>/dev/null &
lbrycrdd -conf=$CONFIG_PATH $1
;;
testnet )
## Set config params
## TODO: Make this more automagic in the future.
mkdir -p `dirname $CONFIG_PATH`
echo "rpcuser=lbry" > $CONFIG_PATH
echo "rpcpassword=lbry" >> $CONFIG_PATH
echo "rpcport=11337" >> $CONFIG_PATH
echo "rpcbind=0.0.0.0" >> $CONFIG_PATH
echo "rpcallowip=0.0.0.0/0" >> $CONFIG_PATH
echo "testnet=1" >> $CONFIG_PATH
echo "txindex=1" >> $CONFIG_PATH
echo "server=1" >> $CONFIG_PATH
echo "printtoconsole=1" >> $CONFIG_PATH
#nohup advance &>/dev/null &
lbrycrdd -conf=$CONFIG_PATH $1
;;
* )
echo "Error, you must define a RUN_MODE environment variable."
echo "Available options are testnet, regtest, chainquery, default, and reindex"
;;
esac
-40
View File
@@ -1,40 +0,0 @@
## This base image is for running latest lbrycrdd
# For some reason I may switch this image over to Alpine when I can RCA why it won't start.
FROM ubuntu:18.04
LABEL MAINTAINER="leopere [at] nixc [dot] us"
RUN addgroup --gid 1000 lbrycrd && \
adduser lbrycrd --uid 1000 --gid 1000 --gecos GECOS --shell /bin/bash --disabled-password --home /data && \
apt-get update && \
apt-get -y install unzip wget curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/*
## TODO: Consider adding debugpaste or variant
# RUN wget -O /usr/bin/debugpaste https://github.com/nixc-us/debugpaste-it/raw/master/bin/debugpaste_64 && \
# chmod +x /usr/bin/debugpaste
RUN wget -O /usr/bin/lbrycrd-linux.zip https://github.com/lbryio/lbrycrd/releases/download/v0.12.3.1/lbrycrd-linux.zip && \
cd /usr/bin/ && \
unzip lbrycrd-linux.zip && \
rm lbrycrd-linux.zip && \
chmod +x lbrycrdd lbrycrd-cli lbrycrd-tx
COPY start.sh /usr/local/bin/start
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
# USER lbrycrd
# RUN mkdir /data
VOLUME ["/data"]
WORKDIR /data
## Exposing daemon port and RPC port
EXPOSE 9245 9246
## TODO: Decide what's important for lbrycrd and possibly add an entrypoint.
## Maybe catch things that might match things that can be easily executed in the
## lbrycrd cli and if nothing is entered just default to the containers shell.
## For now this is a placeholder that executes /bin/bash on `docker exec`
# ENTRYPOINT ["docker-entrypoint"]
CMD ["start"]
-7
View File
@@ -1,7 +0,0 @@
To run a local instance of testnet via docker:
1) clone this repository
2) Run `cd ./lbrycrd/testnet`
3) Run `docker-compose up -d`
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-34
View File
@@ -1,34 +0,0 @@
#!/usr/bin/env bash
# ## ToDo: Get a test case to see if this is the first run or a repeat run.
# ## If it's a first run you need to do a full index including all transactions
# ## tx index creates an index of every single transaction in the block history if
# ## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
# ## This is specific to chainquery.
## Ensure perms are correct prior to running main binary
mkdir -p /data/.lbrycrd
chown -R lbrycrd:lbrycrd /data
chmod -R 755 /data/
## TODO: Consider a config directory for future magic.
# chown -R 1000:1000 /etc/lbrycrd
# chmod -R 755 /etc/lbrycrd
rm -f /var/run/lbrycrd.pid
## Set config params
## TODO: Make this more automagic in the future.
echo "rpcuser=lbry" > /data/.lbrycrd/lbrycrd.conf
echo "rpcpassword=lbry" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcport=11337" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcbind=0.0.0.0" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcallowip=0.0.0.0/0" >> /data/.lbrycrd/lbrycrd.conf
echo "testnet=1" >> /data/.lbrycrd/lbrycrd.conf
echo "txindex=1" >> /data/.lbrycrd/lbrycrd.conf
echo "server=1" >> /data/.lbrycrd/lbrycrd.conf
echo "printtoconsole=1" >> /data/.lbrycrd/lbrycrd.conf
#nohup advance &>/dev/null &
su -c "lbrycrdd -conf=/data/.lbrycrd/lbrycrd.conf" lbrycrd
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
docker build -t tiger5226/testnet:latest .
docker push tiger5226/testnet:latest
+1
View File
@@ -0,0 +1 @@
data/
@@ -0,0 +1,68 @@
## Compiler for lbrynet container for any architecture supported by Ubuntu 18.04
## Specify the BASE_IMAGE build argument to choose which Ubuntu base image to build from.
## Docs for ARG in FROM: https://github.com/docker/cli/blob/master/docs/reference/builder.md#understand-how-arg-and-from-interact
## Architecture | Build command
## x86_64 | docker build -t lbrynet -f Dockerfile-compiler .
## armhf | docker build -t lbrynet-armhf -f Dockerfile-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:armhf-bionic .
## arm64 | docker build -t lbrynet-arm64 -f Dockerfile-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:arm64-bionic .
ARG BASE_IMAGE=ubuntu:18.04
FROM ${BASE_IMAGE} as dependencies
RUN apt-get update && \
apt-get install -y \
python3-pip \
python3.7 \
python3.7-dev \
build-essential \
libssl-dev \
libacl1-dev \
liblz4-dev \
libfuse-dev \
fuse \
pkg-config \
fakeroot \
git \
zlib1g-dev \
libbz2-dev \
libncurses5-dev \
libreadline-dev \
liblzma-dev \
libsqlite3-dev \
zip \
libffi-dev \
libleveldb-dev && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1 && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 10 && \
update-alternatives --config python && \
python --version && \
pip3 --version
FROM dependencies as compile
ARG VERSION=master
RUN python3.7 -m pip install -U pyinstaller && \
git clone https://github.com/lbryio/lbry.git /lbry && \
git -C /lbry checkout ${VERSION}
WORKDIR /lbry
RUN git clone https://github.com/lbryio/torba.git --depth 1 /lbry/torba
WORKDIR /lbry/torba
RUN python3.7 -m pip install -e .
WORKDIR /lbry/
COPY stuff/start.sh /usr/local/bin/start
COPY stuff/checkmount.sh /usr/local/bin/checkmount
RUN python3.7 scripts/set_build.py && \
python3.7 -m pip install -e . && \
pyinstaller -F -n lbrynet lbrynet/extras/cli.py && \
chmod +x /lbry/dist/lbrynet && \
zip -j /lbry/dist/lbrynet-armhf.zip /lbry/dist/lbrynet && \
chmod a+x /usr/local/bin/* && \
mkdir /target && \
/lbry/dist/lbrynet --version
FROM ${BASE_IMAGE} as app
RUN adduser lbrynet --gecos GECOS --shell /bin/bash --disabled-password --home /home/lbrynet
COPY --from=compile /usr/local/bin/start /usr/local/bin/checkmount /lbry/dist/lbrynet /usr/local/bin/
EXPOSE 5279
USER lbrynet
ENTRYPOINT ["/usr/local/bin/checkmount"]
CMD ["start"]
@@ -0,0 +1,30 @@
## This base image is for running the latest lbrynet-daemon release.
FROM ubuntu:18.04
LABEL MAINTAINER="leopere [at] nixc [dot] us"
## Add lbrynet
ADD https://lbry.io/get/lbrynet.linux.zip /lbrynet/lbrynet.linux.zip
RUN apt-get update && apt-get -y install unzip && \
adduser lbrynet --gecos GECOS --shell /bin/bash --disabled-password --home /home/lbrynet && \
unzip /lbrynet/lbrynet.linux.zip -d /lbrynet/ && \
rm /lbrynet/lbrynet.linux.zip && \
chown -Rv lbrynet:lbrynet /lbrynet && \
mv /lbrynet/lbrynet /bin/
COPY stuff/start.sh /usr/local/bin/start
COPY stuff/checkmount.sh /usr/local/bin/checkmount
## Daemon port [Intended for internal use]
## LBRYNET talks to peers on port 3333 [Intended for external use] this port is used to discover other lbrynet daemons with blobs.
## Expose 5566 Reflector port to listen on
## Expose 5279 Port the daemon API will listen on
## the lbryumx aka Wallet port [Intended for internal use]
EXPOSE 4444 3333 5566 5279 50001
## Example daemon_settings.yml is at https://github.com/lbryio/lbry/blob/master/example_daemon_settings.yml
ADD https://raw.githubusercontent.com/lbryio/lbry/master/example_daemon_settings.yml /etc/lbrynet/example_daemon_settings.yml
EXPOSE 5279
USER lbrynet
ENTRYPOINT ["/usr/local/bin/checkmount"]
CMD ["start"]
+61
View File
@@ -0,0 +1,61 @@
# lbrynet
# Docker image tags
`lbry/lbrycrd`
`[linux-x86_64-production](https://github.com/lbryio/lbry-docker/blob/master/chainquery/Dockerfile-linux-x86_64-production)` (Latest release)
## Compiler container
The [Dockerfile-linux-multiarch-compiler](Dockerfile-linux-multiarch-compiler) is for building lbrynet for any architecture supported
by an Ubuntu 18.04 base image.
### Register qemu to run docker images built for platforms other than your host
```
docker run --rm --privileged multiarch/qemu-user-static:register
```
### Build for the default x86_64 platform:
```
docker build -t lbrynet -f Dockerfile-linux-multiarch-compiler .
```
### Build for an ARM 32-bit platform:
```
docker build -t lbrynet-armhf -f Dockerfile-linux-multiarch-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:armhf-bionic .
```
### Build for an ARM 64-bit platform:
```
docker build -t lbrynet-arm64 -f Dockerfile-linux-multiarch-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:arm64-bionic .
```
### Extra build arguments
#### VERSION
Compile any version of lbrynet by specifying the git tag:
```
docker build -t lbrynet:v0.36.0 --build-arg VERSION=v0.36.0 -f Dockerfile-linux-multiarch-compiler .
```
### Running from the compiler container directly
The container requires a home directory to be mounted at `/home/lbrynet`. This
is to ensure that the wallet is backed up to a real storage device. You must run
the container with the appropriate volume argument, or else lbrynet will refuse
to run.
If you compiled lbrynet as above, with the tag `lbrynet-x86`, you could run
docker like so:
```
docker run --rm -it -v wallet:/home/lbrynet lbrynet-x86 lbrynet start
```
This automatically creates a docker volume called `wallet` and it will persist
across container restarts. See more in the [Docker volume
documentation](https://docs.docker.com/storage/volumes/)
@@ -0,0 +1,26 @@
version: '3.4'
networks:
lbry-network:
external: true
services:
#############
## Lbrynet ##
#############
lbrynet:
image: lbry/lbrynet:Linux-x86_64-production
restart: always
networks:
# lbrynet:
# ipv4_address: 10.6.1.4
- lbry-network
ports:
- 4444:4444
- 50001:50001
env_file:
- ../environment/lbrynet.env
## host volumes for persistent data such as wallet private keys.
# This container will not run without a persistent wallet as it may contain a LBC balance.
volumes:
- ../persist/lbrynet/data:/lbrynet
+12
View File
@@ -0,0 +1,12 @@
#!/bin/bash
## TODO: Make a bit more aware of the run mode of this appliance in case there is ever a test mode enabled in the start.sh
mountpoint=/home/lbrynet
if ! grep -qs ".* $mountpoint " /proc/mounts; then
echo "$mountpoint not mounted, refusing to run."
## TODO: We should have documentation that this error references directly with a URL as to why it won't run without a volume.
exit 1
else
`$@`
fi
+2
View File
@@ -0,0 +1,2 @@
#!/bin/bash
lbrynet start
File renamed without changes.