Author SHA1 Message Date
Leopere ebf0a87c98 Fixup 2019-04-28 15:31:26 -04:00
Leopere 027d3872df Updating main README.md to match release status 2019-04-28 15:31:10 -04:00
Leopere eb5427469f Merge branch 'master' into refactor 2019-04-27 18:39:12 -04:00
Leopere de0a21a43c Merge branch 'refactor' of github.com:lbryio/lbry-docker into refactor 2019-04-27 17:52:47 -04:00
Leopere 6ca8e4431a Docker Hub doesn't like self referential markdown urs 2019-04-27 17:52:39 -04:00
Ryan McGuire 2092949655 Updates lbrycrd config file in testnet and regtest modes. 2019-04-27 17:35:40 -04:00
Leopere 10d4edc77d Adding image tags to README's 2019-04-27 17:17:37 -04:00
Leopere 6501267d30 Add chainquery tags 2019-04-27 17:15:23 -04:00
Leopere e8f6fa30d9 Cleaned up documentation
Closes #49
2019-04-27 17:09:49 -04:00
Leopere 0e537ef3e0 Updated documentation 2019-04-27 17:00:34 -04:00
Ryan McGuire 7c6f0b58a1 Adds VERSION ARG to the lbrynet compiler and adds more README. 2019-04-27 17:00:19 -04:00
Leopere 4580d06102 Fixed CONFIG_PATH issue in testing RUN_MODEs 2019-04-27 15:43:01 -04:00
Mark Beamer Jr 58682e1fbe removed permission escalation 2019-04-27 15:42:49 -04:00
Leopere 93ea84bb81 More removals
Renamed to fit with new nomenclature

Removed directory and renamed/moved Dockerfile

Updated COPY path

Moved docker-compose.yml for Lbrynet
Cleaned up comments a bit

Important notice to users in new comment

Updated compose example for image instead of build

updated paths

Added a step for chainquery-bootstrap setup

Moved and updated lbrycrd's env_file

Created lbrynet env_file

Moved and updated chainquery's env_file

Moved and updated lbrynet's compose file

More Cleanup

Added TODO's for checkmount.sh

Renamed compiler

Renamed production container

Final rename

fixup

fixup

Updated README refs

Just readability improvements

Packing directories with the repo

Git won't track empty directories but if you take the path of running these containers locally from within this repo you should have this directory.

Updated image refs to final production locations
2019-04-27 14:37:55 -04:00
Ryan McGuire 44f8f86308 Merges lbrynet compilers into one Dockerfile with ARG for BASE_IMAGE 2019-04-27 14:37:42 -04:00
Leopere c3accd79bd Adjusting to new structure
Replaced build with image docker-compose.yml-prod-example

Adjusted paths in volumes

Removed a few more things

healthcheck.sh which can be added later when completed.
renamed the Dockerfile to Dockerfile-x86_64-production

Updated image tag and added RUN_MODE

Updated image and added RUN_MODE

updated tag

Updated tag
2019-04-27 14:37:40 -04:00
Ryan McGuire 70ae63d02a Adds ENTRYPOINT to lbrynet to require mounted home directory before execution. 2019-04-27 14:37:40 -04:00
Ryan McGuire 69c8468e8f Refactor lbrynet compiler container locations and startup mechanics 2019-04-27 14:37:40 -04:00
Leopere f910732155 Removed traefik labels and moved data dir
Moved data dir and removed traefik label

Moved readme for testnet compose

Adding RUN_MODEs regtest and testnet

Adding advance_blocks.sh into all lbrycrd containers

Adding advance cmd and moved Dockerfile to appropriate dir

Cleanup and moved Dockerfiles

Moved lbrycrd Dockerfiles to appropriate subdirectories and removed.

More moving and deletions

Adding catchall output

Updated default runmode in example

Removed incomplete container directory

This should not be in a production branch at this point now that we have stable containers published.

Removed incomplete container for now

Deleted compile directory and README.md

Cleaning up chainquery

Deleting placeholders

Updated start.sh

Removing superfluous .dockerignores & more

Adjusted start.sh source path

Updated ignores

Updated readme in prep for build tags and ship armhf-prod dir

start.sh's should be recycled when possible.

No sense in code duplication lets try and aim for unification.

Docker prefers WORKDIR over cd

docker prefers WORKDIR over cd
2019-04-27 14:37:38 -04:00
68 changed files with 371 additions and 1120 deletions

No files matched your search

-2
View File
@@ -1,2 +0,0 @@
data/
*/data/
+11 -5
View File
@@ -1,7 +1,13 @@
.gitlab-ci.yml
.DS_Store
*/docker-compose.override.yml
*/*/data
./docker-compose.yml
*.zip
*/docker-compose.override.yml
*/*/data
*.zip
*.tar.gz
*.tar.xz
./persist/lbrycrd/*
./persist/lbrynet/*
./persist/chainquery/*
+49 -28
View File
@@ -1,63 +1,84 @@
# lbry-Docker
## Scope
This repository is in heavy flux as it travels towards [version 1.0](https://github.com/lbryio/lbry-docker/projects/1) however its goal is to make development for and adoption of any of the LBRY appliances trivial. You should be able to clone pull fork your way to a better LBRY without having to do much more than some light reading of a README to get started.
## Tags
<!-- Document tags and link to their Dockerfiles here. -->
#### Documentation is WIP
Currently this repository is a WIP and is under heavy construction, use at your own risk make sure you keep regular backups of your wallets. Your milage may vary as how far this will work for you be sure to file good and concise issues if you plan to and keep in mind we're allergic to regressions when filing PR's.
## Scope
This repository aims to track and maintain most of the Dockerfiles within the LBRY Inc ecosystem.
Currently it supports the following in various form factors:
* Lbrycrd
* Lbrynet
* Chainquery
If you see an architecture that you have a use case for that you don't see in this list and it's stopping you from innovating on the LBRY blockchain search and bump an existing request or create a feature request ASAP on the issue tracker and we will do our best to support your architecture or build requirements.
The goals here will be to create containers that can deploy across a number of technologies namely `docker-compose`, `docker` itself, and Kubernetes k8's.
There will be a few edge cases where this repository will support installing these containers as a service on a machine using `cloud-init` however those come as a YMMV(your mileage may vary) warranty void if used.
We may suggest the use of well commented **`Shell scripts`**, Dockerfiles, and Docker-Compose which is a template that is a baseline for many other container management services such as [RancherOS](https://rancher.com/rancher-os/) and [Kubernetes](https://kubernetes.io/).
#### Goals
This repository aims for [Docker Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) minimalism, ephemerality, and statelessness. It will use well commented **`Shell scripts`**, Dockerfiles, and Docker-Compose which is a template that is a baseline for many other container management services such as [RancherOS](https://rancher.com/rancher-os/) and [Kubernetes](https://kubernetes.io/).
We have a number of priorities within this docker container repository.
Loosely defined our priorities are in the order as follows ordered highest priority to lowest.
1. Congruency - Containers should all behave similarly.
2. Safety - Cautioning the user when they're in a danger zone when possible.
3. Reliability - Only pushing functional tested containers to production allowing users to impose an update schema.
4. Ephemerality/Statelessness - You should be able to define the entire state in a cross cloud platform way such that the container itself should contain no state and thus have no reason to rely on it's specific contents as best as possible.
5. Works by default - As best as possible we will aim for these containers to work by default out of the box then if you wish you can find ways to improve upon what we've set out for the user on your own.
6. Flexibility - Despite all of the above we want people to be able to utilize all of the functionalities of the appliances contained within these containers. This means if you find that our system/implementation gets in your way this is one of our priorities to fix this.
In summary, this repository aims for [Docker Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) minimalism, ephemerality, and statelessness.
#### Beautiful Screenshot/Gif
Since there is nothing to see here just yet I'll entertain you with the current state of affairs with this repository.
![image](https://spee.ch/855d1958650b850b249b9ee592ba2f4c6fc7eeec/container-unloading-gone-wrong-151175.gif)
Here's an example tutorial we have for how to get some of our appliances up and running in minutes on DigitalOcean all hosted on our very own platform https://spee.ch a link to the guide is [here](https://github.com/lbryio/lbry-docker/blob/master/contrib/systemd-cloud-init.md) it contains a few lines of information and some copy paste-able code!
[![Video of creating lbrycrd droplet on DigitalOcean](https://spee.ch/@EnigmaCurry:d/lbrycrd-video-thumb.jpg)](https://spee.ch/@EnigmaCurry:d/lbrycrd-docker-cloud-init.mp4)
## Installation
[This is currently WIP and Not Recommended for Production](https://github.com/lbryio/lbry-docker/projects/1)
See [Running from source](##Running-from-source) for the current instructions on how to use this.
There are a few methods we're planning on releasing over time however two notable options are our [cloud-init](https://github.com/lbryio/lbry-docker/blob/master/contrib/systemd-cloud-init.md) install of lbrycrd and optionally chainquery on DigitalOcean.
Alternatively there is the [shell installer](https://github.com/lbryio/lbry-docker/blob/master/chainquery/README.md) that can run on any Ubuntu 18.04-LTS equipped machine to get you up and running with a database seed which can save you loads of time bootstrapping your instance.
#### Currently supported platforms
**X64 cpu architecture**
**X86_64 CPU architecture**
Lbrynet production & compiler, Lbrycrd, Chainqery are currently supported.
**Some arm architectures**
Lbrynet armhf and arm64 compilers
**More will be added on request and over time**
## Usage
For now I don't recommend using this container cluster however you're welcomed to [contribute](#contributing) if you feel up to the task.
You may see above [the installation section above](https://github.com/lbryio/lbry-docker#installation) to obtain container specific instructions.
## Running from source
Running this stuff from source should be possible if you have both [Docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) both installed. If these are both installed you can proceed to run the following from within your development directory.
Running this stuff from source should be possible if you have both [Docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) both installed. If these are both installed, you can proceed to run the following from within your development directory.
```
git clone https://github.com/lbryio/lbry-docker.git
```
Once you have a local copy of the recent source you will want to consider what containers/applications you require in your environment. At the moment since at the writing of this documentation this comes with an assertion of [YMMV](https://dictionary.cambridge.org/dictionary/english/ymmv) so if something isn't working feel free to make suggestions in the form of a PR for how we should do this better. The beauty of Open Source is learning better ways to do things as well as contributing to the world so I'm always going to be welcoming to contributions.
Once you have a local copy of the recent source, you will want to consider what containers/applications you require in your environment. At the moment since at the writing of this documentation, this comes with an assertion of [YMMV](https://dictionary.cambridge.org/dictionary/english/ymmv) so if something isn't working feel free to make suggestions in the form of a PR for how we should do this better. The beauty of Open Source is learning better ways to do things as well as contributing to the world, so I'm always going to be welcoming to contributions.
#### From Source for Contributions
Running from source for contributing and Merge/Pull requests.
My goal is to make contributing to this possible using Docker and also GitLab CI/CD time.
## [Contributing](CONTRIBUTING.md)
Keep in mind [I am](https://github.com/leopere/) preferential to receiving patches over rule following as we can always nudge you in the right direction to get things more compatible with the project ethos if it's not. Never be afraid to file a PR no one should be offended. This said following the next two guides will greatly improve the speed at which we can integrate your improvements.
Keep in mind [I am](https://github.com/leopere/) preferential to receiving patches over rule-following as we can always nudge you in the right direction to get things more compatible with the project ethos if it's not. Never be afraid to file a PR no one should be offended. Having said this following the next two guides will greatly improve the speed at which we can integrate your improvements.
* [Repository Standards]( https://lbry.tech/resources/repository-standards)
* [Contribute](https://lbry.tech/contribute)
* Have a LBC wallet ready as we want you to have some for the help! Hell why not post it in your Commit or Merge Request for all I care but take your tips!
* Have an LBC wallet ready as we want you to have some for the help! Hell, why not post it in your Commit or Merge Request for all I care but take your tips!
## Getting Support
#### Debugpaste [WIP]
I'll be including a function to get a self destructing debugpaste of your LBRY appliances logs you'll be able to execute something similar to the following in all containers to export raw logs to a paste service where you can then either modify them removing sensitive data or just take that URL and create a new issue after you [(Use Issue Search)](https://github.com/lbryio/lbry-docker/issues?utf8=%E2%9C%93&q=is%3Aissue) to make sure there isn't already an open thread for your issue.
#### Example debugpaste
```
cd chainquery/
docker-compose exec chainquery debugpaste
https://haste.nixc.us/ocatumatozaq.nginx
```
You can then take output given in the response from the debugpaste command and put that into your github issue (This may be automated at some point to some degree).
Sometimes we just aren't omniscient omnipresent or even omnipotent. So since this is the case despite our constant striving for these goals we're aware that things are imperfect so when you find these imperfections please reach out to us! You can become a contributor without even knowing how to code a damn thing! Just open a Git Issue so long as it's not a duplicate its a huge help to us and often times it'll earn you some LBC if you're a part of our community.
#### Creating an Issue
Please be sure to fill out the [issue template](https://github.com/lbryio/lbry-docker/issues/new) as best as possible. This will help us answer your questions faster and fix things quicker if we have quality reports.
-5
View File
@@ -1,5 +0,0 @@
data/
data.z*
chainquery.z*
*.zip
compile/
@@ -8,7 +8,7 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/*
WORKDIR /
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
COPY stuff/start.sh start
COPY ./stuff/start.sh start
RUN curl -L -o /chainquery.zip $(curl -s https://api.github.com/repos/lbryio/chainquery/releases | grep -F 'Linux_x86_64.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
unzip /chainquery.zip && \
rm /chainquery.zip
+30 -21
View File
@@ -1,39 +1,46 @@
# Docker image tags
`lbry/chainquery`
`[linux-x86_64-production](https://github.com/lbryio/lbry-docker/blob/master/chainquery/Dockerfile-linux-x86_64-production)` (Latest release)
# Installation instructions for Chainquery and dependencies
## Preface
I have set up this tutorial so that you should be able to simply copy and paste the commands into your linux console. However if you run into issues please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail.
I have set up this tutorial so that you should be able to copy and paste the commands into your Linux console. However, if you run into issues, please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail.
## Caveats to installing Chainquery
Chainquery is an interface to an extensive index of data which has to parse an entire blockchain's history into a SQL database in a way that is easily query-able. The process of syncing your lbrycrd level.db files can take a long time depending on your deployment and then once your instance is synced you need to index the blockchain with Chainquery. All of this takes a fair bit of time or resources at first it is possible to deploy this stack without the shell scripts, and I will eventually include a docker-compose.yml file which gives you a baseline for deploying this to k8's and other platforms but for now single host deployment via shell script will be what's supported.
## Goals of this README.md
This readme is targeted towards people who need a copy pasta recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine you should be able to see what a running instance of Chainquery looks like and start writing apps against it's API calls etc.
This readme is targeted towards people who need a copy paste recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine, you should be able to see what a running instance of Chainquery looks like and start writing apps against its API.
## Dependencies
My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have will be `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine.
My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have are `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine.
### Docker
Docker
Docker is effectively the cross platform software package repository it allows you to ship an entire environment in what's referred to as a container. Implying that containers hold everything that is needed to ship what's inside effectively from one place to another with a degree of standard that makes it easy for everyone along the way to replicate what is needed for their step in the chain.
The other side of docker is it brings everything that we would normally have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handfull of easy to follow steps that should result in the same environment on everyones host.
Docker is effectively the cross-platform software package repository it allows you to ship an entire environment in what's referred to as a container. Containers are intended to hold everything that is needed to ship what's required to run an application from one place to another with a degree of a standard that makes it easy for everyone along the way to reproduce the environment for their step in the chain.
The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this will suffice. At the writing of this tutorial this was not the latest version of Docker.
The other side of docker is it brings everything that we would typically have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handful of easy to follow steps that should result in the same environment on everyone's host.
#### Just the link to installation instructions please
The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this is sufficient. At the writing of this tutorial, this was not the latest version of Docker.
#### Just the link to installation instructions, please
Instructions for installing on Ubuntu are at the link that follows:
https://docs.docker.com/install/linux/docker-ce/ubuntu/
If you plan on using other versions of OS you should at least aim to be a linux base with a x64 CPU and the appropriate minimum version of the dependencies.
If you plan on using other versions of OS you should at least aim to be a Linux base with an x86_64 CPU and the appropriate minimum version of the dependencies.
### Docker-compose
Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format but it's not a required prerequisite for getting this running you just need to have it installed successfully.
Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format, but it's not a required prerequisite for getting this running you need to have it installed successfully.
Install Docker Compose via this guide below, its important if you're using an older version of linux to use the official documentation from Docker.com because you will need the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer.
Install Docker Compose via this guide below, and it is essential if you're using an older version of Linux to use the official documentation from Docker.com because you require the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer.
#### Just the link to installation instructions please
#### Just the link to installation instructions, please
https://docs.docker.com/compose/install/
### Git
For now the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally fairly minimal.
For now, the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally reasonably minimal.
#### Just the instructions please
#### Just the instructions, please
`apt-get install git -y`
## Get the lbry-docker repository
@@ -43,14 +50,14 @@ For now the recommended install includes grabbing the latest git repository from
## Setup networking
You only need external networking if you plan on keeping your docker-compose files separate.
For the sake of modularity in the design of this git repository the plan is to give you examples to try and then you're supposed to move towards your own custom docker-compose configuration. We're going to create
a docker bridge network that is going to be managed externally to your usual docker-compose networks which are compose internal.
For the sake of modularity in the design of this git repository, the plan is to give you examples to try and then you're supposed to move towards your custom docker-compose configuration. We're going to create
a docker bridge network that is going to be managed externally to your usual docker-compose networks which are docker-compose internal.
`docker network create -d bridge --subnet=10.6.1.0/16 lbry-network`
## Make directories and set permissions
This is only required on host mounted volumes. (the default settings)
Setting permissions is only typically required on host mounted volumes. (the default settings)
`sudo mkdir -p ./lbry-docker/lbrycrd/data`
@@ -66,17 +73,19 @@ This is only required on host mounted volumes. (the default settings)
Wait for lbrycrd to reach the top of the blockchain (console output should noticeably slow down)
Once you've reached the top of the blockchain you can press `CTRL+C` to exit back to the linux shell.
Once you've reached the top of the blockchain, you can press `CTRL+C` to exit back to the Linux shell.
## Setup chainquery
Now that you're done syncing your own copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery and it's dependencies. Since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go.
Now you're finished syncing your copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery, and it's dependencies since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go.
1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using a LBRY.io provided database checkpoint snapshot. This should be basically copy paste-able series of commands to get your own staging instance up asap if you want Route number one, follow along with the rest of this README.md
2. Route number two: You can your own fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware.
1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using an LBRY.io provided database checkpoint snapshot. The checkpoint should be copy-paste-able series of commands to get your staging instance up asap if you want Route number one, follow along with the rest of this README.md
2. Route number two: You can your fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware.
`cd ../chainquery/`
`cat ./compose/docker-compose.yml-prod-example > docker-compose.yml`
`./quick-bootstrap.sh getdata`
`./quick-bootstrap.sh extract`
-4
View File
@@ -1,4 +0,0 @@
data/
data.z*
chainquery.z*
compile/
-16
View File
@@ -1,16 +0,0 @@
COMPOSE_PROJECT_NAME=chainquery
#########################
## Chainquery Settings ##
#########################
RPC_ALLOW_IP=10.5.1.3
DEBUGMODE=false
#################
## Mysql Creds ##
#################
MYSQL_SERVER=10.5.1.10
MYSQL_USER=changeme
MYSQL_PASSWORD=changeme
MYSQL_DATABASE=chainquery
MYSQL_ROOT_PASSWORD=changeme
-4
View File
@@ -1,4 +0,0 @@
data/
data.z*
chainquery.z*
compile/
-52
View File
@@ -1,52 +0,0 @@
## This base image is for running latest chainquery
FROM ubuntu:18.04 as prep
LABEL MAINTAINER="leopere [at] nixc [dot] us"
## Install everything needed to unzip the file containing Chainquery
RUN apt-get update && \
apt-get -y install unzip curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/* && \
mkdir /download
## Download and extract the latest Zip containing Linux binary for Chainquery.
RUN curl -L -o /download/chainquery.zip $(curl -s https://api.github.com/repos/lbryio/chainquery/releases | fgrep 'Linux_x86_64.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
cd /download && \
ls -lAh /download && pwd && \
unzip chainquery.zip && \
rm chainquery.zip
## TODO: Use this instead of everything above here at some point. To do this we will need the LBRY team to host all of their release binaries in their latest form internally I guess since github doesn't support latest tags.
# ADD --chown=1000:1000 https://github.com/lbryio/chainquery/releases/download/latest/chainquery_latest_Linux_x86_64.zip /download
# TODO: Switch to Alpine eventually we might need to compile in Alpine for this.
FROM ubuntu:18.04 as app
## Run as unprivileged user.
RUN adduser chainquery --gecos GECOS --shell /bin/bash --disabled-password --home /home/chainquery
# Pull chainquery executable from prep container.
COPY --from=0 /download/chainquery /usr/bin
# ADD --chown=1000:1000 https://github.com/lbryio/chainquery/releases/download/latest/chainquery_latest_Linux_x86_64.zip /usr/bin
# COPY stuff/chainqueryconfig.toml /etc/chainquery/
## Get latest chainqueryconfig.toml from master on github as template.
# TODO: add magic which pattern matches for chainqueryconfig.toml DEFAULT strings and replaces them with configured settings on container start.
ADD https://raw.githubusercontent.com/lbryio/chainquery/master/config/default/chainqueryconfig.toml /etc/chainquery/chainqueryconfig.toml.orig
## Install start.sh as executable script in container $PATH
COPY stuff/start.sh /usr/local/bin/start
# TODO: Implement docker-entrypoint if later required this might be handy if we need certain maintainence tasks to be executed in the live container.
# COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
# ## From here onward we're doing this with no root.
# USER chainquery
## Expose Chainquery API port
EXPOSE 6300
## Execute start script earlier installed into $PATH
CMD ["start"]
-57
View File
@@ -1,57 +0,0 @@
version: '3.4'
networks:
lbrynet:
external: true
services:
###########
## MYSQL ##
###########
## MariaDB is currently not supported and neither is later versions of MySQL this may change.
## https://hub.docker.com/r/_/mariadb/
mysql:
image: mysql:5.7.23
restart: always
networks:
lbrynet:
ipv4_address: 10.5.1.10
aliases:
- mysql
env_file:
- .env
expose:
- 3306
## TODO: I want to find a way that is acceptable to everyone to lock this up
## and not share it with everyone at least eventually.
ports:
- 3306:3306
volumes:
- ./data/db:/var/lib/mysql
- ./stuff/my.cnf:/etc/mysql/conf.d/chainquery-optimizations.cnf
################
## Chainquery ##
################
chainquery:
build:
context: .
target: app
restart: always
networks:
lbrynet:
ipv4_address: 10.5.1.3
env_file:
- .env
- ../lbrycrd/.env
labels:
- "traefik.expose=false"
expose:
- 6300
ports:
- 6300:6300
depends_on:
- mysql
## TODO: Uncomment this in a docker-compose.override.yml to allow for external configurations.
# volumes:
# - ./data/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml
-148
View File
@@ -1,148 +0,0 @@
#!/usr/bin/env bash
## TODO: Be Polite and ask for confirmation.
function QandA() {
read -r -p "Continue with $1 [y/N] " response
response=${response,,} # tolower
if [[ "$response" =~ ^(yes|y)$ ]]; then
echo "Continuing with this."
eval $1
else
echo "Skipping the $1."
fi
}
## Check your $PATH for required dependencies.
## Stop and complain for now, later automagically install them.
## TODO: Add dependency checker.
function test_for_deps() {
## Test for Command
if ! [ -x "$(command -v $1)" ]; then
echo "Error: $1 is not installed." >&2
echo "You must have $1 installed."
else
echo "Info: $1 is installed."
fi
}
## Declare Linux app dependencies to check for.
DEPENDENCIES=(
docker
docker-compose
)
## TODO: Check for docker and docker-compose
function check_deps() {
for i in "${!DEPENDENCIES[@]}"; do
echo ${DEPENDENCIES[$i]}"_KEY"
## Indirect references http://tldp.org/LDP/abs/html/ivr.html
eval TESTDEP=\$"${DEPENDENCIES[$i]}"
test_for_deps $TESTDEP
done
}
function get_checkpoint() {
## Get DB Checkpoint data.
echo Asked to get the latest checkpoint data, downloading latest checkpoint.
echo This data is fairly large so this saves you a few days of parsing the LBRY blockchain.
docker run -v $(pwd)/:/download --rm leopere/axel-docker http://chainquery-data.s3.amazonaws.com/chainquery-data.zip -o ./chainquery.zip
}
#################################
## The real action begins here ##
#################################
## TODO: Add ways to get into and out of a bind here.
case $1 in
getdata )
if [[ -f ./chainquery.zip ]]; then
echo "Found a copy of ./chainquery.zip already in your system."
echo "We recommend that you delete this data before proceeding and grab a fresh copy."
QandA "rm -f ./chainquery.zip"
get_checkpoint
else
get_checkpoint
fi
;;
extract )
## Unpack the data again if need be.
echo Asked to unpack chainquery.zip if downloaded.
if [[ -f ./chainquery.zip ]]; then
docker run -v $(pwd)/:/data --rm leopere/unzip-docker ./chainquery.zip
else
echo "Could not extractas chainquery.zip did not exist."
echo "Feel free to execute './quick-bootstrap.sh getdata' first next time."
fi
;;
cleanup )
## Remove any junk here.
echo Asked to clean up leftover chainquery.zip to save on disk space.
rm chainquery.zip
;;
reset )
## Give up on everything and try again.
## TODO: Make it very obvious with a nice little Y/N prompt that you're about to trash your settings and start over.
echo "Agressively Killing all chainquery and dependency containers."
echo "executing: docker-compose kill"
docker-compose kill
echo "Cleaning up stopped containers."
echo "executing: docker-compose rm -f"
docker-compose rm -f
rm -Rf ./data
rm -f ./chainquery.zip
;;
start )
## Unsupported start command to start containers.
## You can use this if you want to start this thing gracefully.
## Ideally you would not use this in production.
echo "Asked to start chainquery gracefully for you."
echo "executing: docker-compose up -d mysql"
docker-compose up -d mysql
echo "giving mysql some time to establish schema, crypto, users, permissions, and tables"
sleep 30
echo "Starting Chainquery"
echo "executing: docker-compose up -d chainquery"
docker-compose up -d chainquery
## TODO: verify chainquery instance is up and healthy, this requires a functional HEALTHCHECK
echo "This should have chainquery up and running, currently theres no checks in this function to verify this however."
echo "Do feel free to execute 'docker-compose ps' to verify if its running and not restarting or exited."
echo "Final Note: You should try to use the docker-compose commands in the tutorial at https://github.com/lbryio/lbry-docker/blob/master/chainquery/README.md"
;;
compress-latest-checkpoint-data )
## This is not intended for public use.
docker-compose stop chainquery
docker-compose stop mysql
sudo zip -r chainquery-data.zip data
docker-compose up -d mysql
sleep 30
docker-compose up -d chainquery
;;
upload-latest-checkpoint-data )
## This is not intended for public use.
aws s3 cp ./chainquery-data.zip s3://chainquery-data/chainquery-data.new
aws s3 rm s3://chainquery-data/chainquery-data.zip
aws s3 mv s3://chainquery-data/chainquery-data.new s3://chainquery-data/chainquery-data.zip
;;
* )
echo "=================================================="
echo "You look like you need usage examples let me help."
echo "=================================================="
echo "./quick-boostrap.sh {Parameter}"
echo "Example: ./quick-bootstrap.sh getdata # Downloads the latest Chainquery checkpoint data from a LBRYio official aws instance."
echo ""
echo ""
echo "=================================================="
echo "Usage example and available parameters"
echo "=================================================="
echo ""
echo "getdata # This function grabs the latest Chainquery checkpoint data."
echo "extract # Unpacks the chainquery data into the correct directory. ./data/"
echo "cleanup # Removes chainquery.zip"
echo "reset # Reset the state of these containers entirely, use if all else fails."
echo ""
echo ""
echo "=================================================="
echo "=================================================="
echo "Any other functions that are not documented here are not intended for public use."
echo " These functions are included in this repository to keep things in one place."
;;
esac
@@ -1,16 +0,0 @@
## TODO: Don't hardcode this stuff for production
#Debug mode outputs specific information to the console
debugmode=false
#LBRYcrd URL is required for chainquery to query the blockchain
lbrycrdurl="rpc://lbryrpc:changeme@10.5.1.2:9245"
#MySQL DSN is required for chainquery to store information.
mysqldsn="changeme:changeme@tcp(10.5.1.10:3306)/chainquery"
#API MySQL DSN is required for chainquery to expose a SQL query service
apimysqldsn="changeme:changeme@tcp(10.5.1.10:3306)/chainquery"
#The command that should be executed to trigger a self update of the software. For linux, for example, `<yourscript>.sh`
#DEFAULT-autoupdatecommand=[unset]
@@ -1 +0,0 @@
#!/usr/bin/env bash
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-19
View File
@@ -1,19 +0,0 @@
COMPOSE_PROJECT_NAME=chainquery
#########################
## Chainquery Settings ##
#########################
## TODO: Test to ensure these vars can be pulled from the lbrycrd .env
#RPC_USER=lbryrpc ## Not super necessery to change this.
#RPC_PASSWORD=changeme ## Please replace changeme.
RPC_ALLOW_IP=10.5.1.3 ## You're better off not changing this.
#################
## Mysql Creds ##
#################
MYSQL_SERVER=10.5.1.10 ## You're better off not changing this.
MYSQL_USER=changeme ## This could be changed.
MYSQL_PASSWORD=changeme ## This could be set to something random it sets this string for both Mysql's main user and Chainquery's MysqlDSN.
MYSQL_DATABASE=chainquery ## This can stay the same.
MYSQL_ROOT_PASSWORD=changeme ## Set this to something random and obnoxious we're not using it.
-3
View File
@@ -1,3 +0,0 @@
#!/usr/bin/env bash
## TODO: Implement this at some point it requires CURL in the container.
curl http://localhost:6300/api/status
-9
View File
@@ -1,9 +0,0 @@
# Default Homebrew MySQL server config
[mysqld]
# Only allow connections from localhost
innodb_log_file_size=5G
key_buffer_size=1G
innodb_flush_log_at_trx_commit = 0
innodb_autoinc_lock_mode=2
innodb_buffer_pool_size=1G
innodb_log_buffer_size=1G
-54
View File
@@ -1,54 +0,0 @@
#!/usr/bin/env bash
## Config setup
## Setup Values
DEBUGMODE=$(echo "debugmode=$DEBUGMODE")
LBRYCRDURL=$(echo "lbrycrdurl=\"rpc://$RPC_USER:$RPC_PASSWORD@10.5.1.2:9245\"")
MYSQLDSN=$(echo "mysqldsn=\"$MYSQL_USER:$MYSQL_PASSWORD@tcp($MYSQL_SERVER:3306)/$MYSQL_DATABASE\"")
APIMYSQLDSN=$(echo "apimysqldsn=\"$MYSQL_USER:$MYSQL_PASSWORD@tcp($MYSQL_SERVER:3306)/$MYSQL_DATABASE\"")
## Setup Defaults
DEBUGMODE_DEFAULT='#DEFAULT-debugmode=false'
LBRYCRDURL_DEFAULT='#DEFAULT-lbrycrdurl="rpc://lbry:lbry@localhost:9245"'
MYSQLDSN_DEFAULT='#DEFAULT-mysqldsn="lbry:lbry@tcp(localhost:3306)/chainquery"'
APIMYSQLDSN_DEFAULT='#DEFAULT-apimysqldsn="lbry:lbry@tcp(localhost:3306)/chainquery"'
## Add setup value variable name to this list to get processed on container start
CONFIG_SETTINGS=(
DEBUGMODE
LBRYCRDURL
MYSQLDSN
APIMYSQLDSN
)
function set_configs() {
## Set configs on container start if not already set.
for i in "${!CONFIG_SETTINGS[@]}"; do
## Indirect references http://tldp.org/LDP/abs/html/ivr.html
eval FROM_STRING=\$"${CONFIG_SETTINGS[$i]}_DEFAULT"
eval TO_STRING=\$${CONFIG_SETTINGS[$i]}
## TODO: Add a bit more magic to make sure that you're only configuring things if not set by config mounts.
sed -i "s~$FROM_STRING~"$TO_STRING"~g" /etc/chainquery/chainqueryconfig.toml
done
echo "Reading config for debugging."
cat /etc/chainquery/chainqueryconfig.toml
}
if [[ ! -f /etc/chainquery/chainqueryconfig.toml ]]; then
echo "[INFO]: Did not find chainqueryconfig.toml"
echo " Installing default and configuring with provided environment variables if any."
## Install fresh copy of config file.
echo "cp -v /etc/chainquery/chainqueryconfig.toml.orig /etc/chainquery/chainqueryconfig.toml"
cp -v /etc/chainquery/chainqueryconfig.toml.orig /etc/chainquery/chainqueryconfig.toml
chmod 755 /etc/chainquery/chainqueryconfig.toml
ls -lAh /etc/chainquery/
set_configs
else
echo "[INFO]: Found a copy of chainqueryconfig.toml in /etc/chainquery"
echo " Attempting to non destructively install any new environment configurations."
set_configs
fi
## For now keeping this simple. Potentially eventually add all command args as envvars for the Dockerfile or use safe way to add args via docker-compose.yml
su -c "chainquery serve -c "/etc/chainquery/"" chainquery
@@ -19,7 +19,7 @@ services:
aliases:
- mysql
env_file:
- env
- ../environment/chainquery.env
expose:
- 3306
## TODO: I want to find a way that is acceptable to everyone to lock this up
@@ -27,23 +27,22 @@ services:
ports:
- 3306:3306
volumes:
- ./data/db:/var/lib/mysql
- ./stuff/my.cnf:/etc/mysql/conf.d/chainquery-optimizations.cnf
- ../persist/chainquery/db:/var/lib/mysql
## This one may need to be tweaked based on where you run this docker-compose from.
- ../stuff/my.cnf:/etc/mysql/conf.d/chainquery-optimizations.cnf
################
## Chainquery ##
################
chainquery:
build:
context: .
target: app
image: lbryio/lbry-docker:linux-x86_64-production
restart: always
networks:
lbry-network:
ipv4_address: 10.6.1.3
env_file:
- env
- ../lbrycrd/env
- ../environment/lbrycrd.env
labels:
- "traefik.expose=false"
expose:
@@ -53,5 +52,5 @@ services:
depends_on:
- mysql
## TODO: Uncomment this in a docker-compose.override.yml to allow for external configurations.
# volumes:
# - ./data/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml
volumes:
- ../persist/chainquery/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml
-1
View File
@@ -1 +0,0 @@
#!/usr/bin/env bash
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
+32 -33
View File
@@ -4,18 +4,18 @@ Contributing Author: [EnigmaCurry](https://www.enigmacurry.com)
Last Update: April 18 2019
This is meant to be easy instructions for running a lbrycrd and chainquery
This tutorial meant to be easy instructions for running a lbrycrd and chainquery
service on DigitalOcean. It's pretty much just copy-and-paste.
This should also work on any host that supports
This tutorial should also work on any host that supports
[cloud-init](https://cloud-init.io/), but I've not tested it anywhere except for
DigitalOcean.
If you wish to use docker-compose, there is an [alternative
configuration](https://github.com/lbryio/lbry-docker/tree/master/lbrycrd)
for that. This tutorial will use cloud-init and systemd to control docker.
for that. This tutorial uses cloud-init and systemd to control docker.
## It's easy to run your own full lbrycrd node
## It's easy to run your full lbrycrd node
[![Video of creating lbrycrd droplet on DigitalOcean](https://spee.ch/@EnigmaCurry:d/lbrycrd-video-thumb.jpg)](https://spee.ch/@EnigmaCurry:d/lbrycrd-docker-cloud-init.mp4)
@@ -26,15 +26,15 @@ for that. This tutorial will use cloud-init and systemd to control docker.
* Select a Standard droplet with 8GB of memory ($40 per month in 2019.)
* Select whatever datacenter you want.
* Mark the checkbox called `User data`, and paste the following into the box:
```
#cloud-config
## DigitalOcean user-data for Ubuntu 18.04 droplet
## Installs docker
## Setup systemd service for lbrycrd
## (This config just runs docker on vanilla Ubuntu,
## it uses systemd inplace of docker-compose or kubernetes.)
## (This config runs docker on vanilla Ubuntu,
## it uses systemd in place of docker-compose or kubernetes.)
write_files:
- path: "/etc/lbry/lbrycrd.conf"
@@ -57,10 +57,10 @@ write_files:
Description=lbrycrd docker container
After=snap.docker.dockerd.service
Requires=snap.docker.dockerd.service
[Service]
Environment=SERVICE=lbrycrd
Environment=IMAGE=lbry/lbry-docker:lbrycrd-production
Environment=IMAGE=lbry/lbrycrd:linux-x86-64-production
TimeoutStartSec=0
ExecStartPre=-/snap/bin/docker stop $SERVICE
ExecStartPre=-/snap/bin/docker rm -f $SERVICE
@@ -77,14 +77,14 @@ write_files:
ExecStop=/snap/bin/docker stop $SERVICE
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
- path: "/etc/mysql/conf.d/chainquery.cnf"
content: |
# Put mysql optimizations specific to chainquery here
# Put MySQL optimizations specific to chainquery here
- path: "/etc/systemd/system/mysql.service"
content: |
[Unit]
@@ -112,7 +112,7 @@ write_files:
ExecStop=/snap/bin/docker stop $SERVICE
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
@@ -122,7 +122,7 @@ write_files:
lbrycrdurl="rpc://lbry:lbry@lbrycrd:9245"
mysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
apimysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
- path: "/etc/systemd/system/chainquery.service"
content: |
[Unit]
@@ -133,7 +133,7 @@ write_files:
[Service]
Environment=SERVICE=chainquery
Environment=IMAGE=lbry/lbry-docker:chainquery-production
Environment=IMAGE=lbry/chainquery:linux-x86-64-production
TimeoutStartSec=0
ExecStartPre=-/snap/bin/docker stop $SERVICE
ExecStartPre=-/snap/bin/docker rm -f $SERVICE
@@ -149,7 +149,7 @@ write_files:
ExecStop=/snap/bin/docker stop $SERVICE
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
@@ -157,7 +157,7 @@ write_files:
- path: "/root/.bash_aliases"
content: |
alias lbrycrd-cli="docker run --rm -it --link lbrycrd:lbrycrd --mount type=bind,source=/etc/lbry/lbrycrd.conf,target=/etc/lbry/lbrycrd.conf \
lbry/lbry-docker:lbrycrd-production lbrycrd-cli -conf=/etc/lbry/lbrycrd.conf -rpcconnect=lbrycrd"
lbry/lbrycrd:linux-x86-64-production lbrycrd-cli -conf=/etc/lbry/lbrycrd.conf -rpcconnect=lbrycrd"
alias mysql="docker run --rm -it --link mysql:mysql mysql:5 mysql -hmysql -u chainquery --password=chainquery"
runcmd:
@@ -169,15 +169,15 @@ runcmd:
- /snap/bin/docker volume create mysql-data
- systemctl enable --now lbrycrd
- echo "Good to go."
```
```
* You can leave everything above as it is, to use the default configuration, OR
you may edit the config in the box to your own liking.
you may edit the config in the box to your liking.
* For instance, if you wanted to run in [regtest
mode](https://lbry.tech/resources/regtest-setup), you would set
`regtest=1` in the first section under `write_files`.
* You can also edit the config files at any later point in `/etc/lbry`,
after you create the droplet.
* Select your SSH key so you can login.
* Select your SSH key so you can log in.
* Give it a good hostname.
* Click Create.
@@ -186,7 +186,7 @@ runcmd:
### How to administer the system
Copy the IP address from the droplet status page, SSH into the droplet as root
using the same SSH key you configured for the droplet.
using the same SSH key, you configured for the droplet.
The config file is in `/etc/lbry/lbrycrd.conf` on the host.
@@ -199,7 +199,7 @@ startup.
You can tail the log to monitor the install progress:
```
tail -f /var/log/cloud-init-output.log
tail -f /var/log/cloud-init-output.log
```
Wait for the final `Good to go` message to know that the installer has finished.
@@ -208,7 +208,7 @@ Wait for the final `Good to go` message to know that the installer has finished.
#### Check the status of the systemd service
You can interact with systemd using `systemctl` (status, start, stop, restart,
etc.) and `journalctl` (logging) tools.
and more.) and `journalctl` (logging) tools.
```
systemctl status lbrycrd
@@ -235,7 +235,7 @@ docker logs lbrycrd
### Utilize lbrycrd-cli
You can use lbrycrd-cli from the host console. A bash alias has been added to
You can use lbrycrd-cli from the host console. A bash alias gets added to
`/root/.bash_aliases` that invokes lbrycrd-cli in its own container.
```
@@ -261,28 +261,28 @@ $ lbrycrd-cli getinfo
### Chainquery (optional)
The chainquery service is pre-installed, but it is not enabled by default.
The chainquery service is pre-installed but not enabled by default.
#### Enable and start the mysql service
#### Enable and start the MySQL service
```
systemctl enable --now mysql
```
In case you need it, there is a bash alias called `mysql`
(`/root/.bash_aliases`) for the mysql client that allows you to login to the
In case you need it, there is a bash alias called `mysql`.
(`/root/.bash_aliases`) for the MySQL client that allows you to log in to the
chainquery database.
#### Enable and start the chainquery service
The chainquery config file is located on the host: `/etc/lbry/chainqueryconfig.toml`
The chainquery config file located on the host at: `/etc/lbry/chainqueryconfig.toml`
```
systemctl enable --now chainquery
```
In systemd, when you enable a service, it means to always start the service at
system boot. (`--now` just means you also want to start the service right away.)
In systemd, when you enable a service, it means always to start the service at
system boot. (`--now` means you also want to start the service right away.)
As with any service, you can control chainquery with `systemctl` and get logs
with `journalctl`:
@@ -304,7 +304,7 @@ journalctl --unit chainquery
(optionally use `-f` if you want to tail/follow the logs)
##### Disabling chainquery service
##### Disabling chainquery service
```
systemctl disable --now chainquery
@@ -319,4 +319,3 @@ In the future, this tutorial may replace the snap version of docker with the
regular PPA version of docker-ce, which has a more predictable update strategy
(apt-get) rather than auto-updates. More long term testing is needed to know
which way is better.
@@ -1,4 +1,4 @@
COMPOSE_PROJECT_NAME=chainquery
## Contains environment variables for chainquery containers
#########################
## Chainquery Settings ##
+1 -2
View File
@@ -1,5 +1,4 @@
COMPOSE_PROJECT_NAME=lbrycrd
## Contains environment variables for lbrycrd containers
#############
## Lbrycrd ##
#############
+2
View File
@@ -0,0 +1,2 @@
## Contains environment variables for lbrynet containers
RUN_MODE=default
-1
View File
@@ -1 +0,0 @@
data/
@@ -7,18 +7,21 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/*
WORKDIR /
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
COPY stuff/start.sh start
COPY stuff/healthcheck.sh healthcheck
COPY stuff/fix-permissions.c fix-permissions.c
COPY ./stuff/start.sh start
COPY ./stuff/healthcheck.sh healthcheck
COPY ./stuff/advance_blocks.sh advance
COPY ./stuff/fix-permissions.c fix-permissions.c
RUN curl -L -o ./lbrycrd-linux.zip $(curl -s https://api.github.com/repos/lbryio/lbrycrd/releases | grep -F 'lbrycrd-linux.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
unzip ./lbrycrd-linux.zip && \
gcc fix-permissions.c -o fix-permissions && \
chmod +x ./lbrycrdd ./lbrycrd-cli ./lbrycrd-tx ./start ./healthcheck ./fix-permissions
chmod +x ./lbrycrdd ./lbrycrd-cli ./lbrycrd-tx ./start ./healthcheck ./fix-permissions ./advance
FROM ubuntu:18.04 as app
COPY --from=prep /lbrycrdd /lbrycrd-cli /lbrycrd-tx /start /healthcheck /fix-permissions /usr/bin/
COPY --from=prep /lbrycrdd /lbrycrd-cli /lbrycrd-tx /start /healthcheck /fix-permissions /advance /usr/bin/
RUN addgroup --gid 1000 lbrycrd && \
adduser lbrycrd --uid 1000 --gid 1000 --gecos GECOS --shell /bin/bash --disabled-password --home /data && \
mkdir /etc/lbry && \
chown lbrycrd /etc/lbry && \
chmod a+s /usr/bin/fix-permissions
VOLUME ["/data"]
WORKDIR /data
+4
View File
@@ -0,0 +1,4 @@
# lbrycrd
# Docker image tags
`lbry/lbrycrd`
`[linux-x86_64-production](https://github.com/lbryio/lbry-docker/blob/master/lbrycrd/Dockerfile-linux-x86_64-production)` (Latest release)
-1
View File
@@ -1 +0,0 @@
data/
-67
View File
@@ -1,67 +0,0 @@
###########################
## Compiler/Builder Step ##
###########################
FROM ubuntu:18.04 as builder
## TODO: Decide who gets the MAINTAINER label
## LABEL MAINTAINER="@Brannon Update Me Or Put Mine Here"
## Added apt work as single container layer then remove cruft before layer is committed.
RUN apt-get update && \
apt-get install -y build-essential libtool autotools-dev automake pkg-config bsdmainutils python3 git wget apt-utils && \
apt-get install -y libssl-dev libevent-dev libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libboost-locale-dev libboost-program-options-dev && \
apt-get install -y python-dev libbz2-dev autoconf ca-certificates clang-format && \
apt-get install -y software-properties-common && add-apt-repository -y ppa:bitcoin/bitcoin && apt-get update && apt-get install -y libdb4.8-dev libdb4.8++-dev && \
apt-get -y install unzip wget && \ ## Cleanup Post apt package installation.
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/* ## Others neednt play here(remove apt lists as no longer needed at this stage).
## Add debugpaste here for reporting compiler errors. Psuedo code.
## RUN wget -O /usr/local/bin/debugpaste https://url && \
## sha256sum check of debugpaste
########################
## Compile Steps Here ##
########################
## Compile and Test steps should likely be separate run commands/layers.
RUN ## Compile/Test steps here
####################
## App build step ##
####################
FROM ubuntu:18.04 as app
## Change this to pull from a certifiable source with supply chain in plain sight.
# RUN wget -O /usr/bin/debugpaste https://github.com/nixc-us/debugpaste-it/raw/master/bin/debugpaste_64 && \
# chmod +x /usr/bin/debugpaste
RUN adduser lbrycrd --gecos GECOS --shell /bin/bash --disabled-password --home /app
# ## This section can be replaced with a COPY to pull artifacts from build step.
# # RUN wget -O /usr/bin/lbrycrd-linux.zip https://github.com/lbryio/lbrycrd/releases/download/v0.12.2.2/lbrycrd-linux.zip && \
# # cd /usr/bin/ && \
# # unzip lbrycrd-linux.zip && \
# # rm lbrycrd-linux.zip && \
# # chmod +x lbrycrdd lbrycrd-cli lbrycrd-tx
# COPY --chown=1000:1000 /path/to/compiled/lbrycrd/bins /app/
## This needn't be included until debugpaste is ready.
# COPY debugpaste-it.sh /usr/local/bin/debugpaste-it
# COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
COPY start.sh /usr/local/bin/start
USER lbrycrdd
RUN mkdir -p /data/
VOLUME /data/
VOLUME /etc/lbrycrdd/
## Exposing daemon port and RPC port
EXPOSE 9245 9246
# ## For now this is a placeholder that executes /bin/bash on `docker exec`
# ENTRYPOINT ["docker-entrypoint"]
CMD ["start"]
View File
Whitespace-only changes.
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-56
View File
@@ -1,56 +0,0 @@
#!/usr/bin/env bash
# ## ToDo:
# ## Get a test case to see if this is the first run or a repeat run
# ## If it's a first run you need to do a full index including all transactions
# ## tx index creates an index of every single transaction in the block history if
# ## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
# ## This is specific to chainquery.
## Ensure perms are correct prior to running main binary
mkdir -p /data/.lbrycrd
chown -R 1000:1000 /data
chmod -R 755 /data
# chown -R 1000:1000 /etc/lbrycrdd
# chmod -R 755 /etc/lbrycrdd
rm -f /var/run/lbrycrdd.pid
## Set config params
echo 'rpcuser=lbryrpc\nrpcpassword='${RPC_PASSWORD:-changeme} > ~/.lbrycrd/lbrycrd.conf
echo "rpcallowip=${RPC_ALLOW_IP:-10.6.1.3}" >> ~/.lbrycrd/lbrycrd.conf
echo "rpcuser=${RPC_USER:-lbryrpc}" >> ~/.lbrycrd/lbrycrd.conf
## Control this invocation through envvar.
case ${RUN_MODE:-default} in
default )
lbrycrdd \
-server \
-conf=$HOME/.lbrycrd/lbrycrd.conf \
-printtoconsole
;;
reindex )
lbrycrdd \
-server \
-txindex \
-reindex \
-conf=$HOME/.lbrycrd/lbrycrd.conf \
-printtoconsole
;;
chainquery )
lbrycrdd \
-server \
-txindex \
-conf=$HOME/.lbrycrd/lbrycrd.conf \
-printtoconsole
;;
esac
## We were unsure if these function as intended so they were disabled for the time being.
# -port=${PORT:-9246} \
# -data=${DATA_DIR:-/data/} \
# -pid=${PID_FILE:/var/run/lbrycrdd.pid} \
# -rpcport=${RPC_PORT:-9245} \
# -rpcpassword=${RPC_PASSWORD:-changeme} \
# -rpcuser=${RPC_USER:-lbryrpc} \
# -rpcallowip=${RPC_ALLOW_IP:-10.6.1.3}
@@ -14,10 +14,8 @@ services:
networks:
lbry-network:
ipv4_address: 10.6.1.2
labels:
- "traefik.expose=false"
environment:
RUN_MODE: chainquery
RUN_MODE: default
env_file:
- env
expose:
@@ -25,4 +23,4 @@ services:
- 9246
## host volumes for persistent data such as wallet private keys.
volumes:
- "./data:/data"
- "../persist/data:/data"
@@ -5,13 +5,13 @@ services:
## Lbrycrd ##
#############
lbrycrd:
image: tiger5226/regtest
image: lbry/lbrycrd:linux-x86_64-production
restart: always
labels:
- "traefik.expose=true"
ports:
- "11336:9246"
- "11337:11337"
## host volumes for persistent data such as wallet private keys.
volumes:
- "./data:/data"
- "../persist/data:/data"
environment:
- RUN_MODE=regtest
@@ -5,13 +5,13 @@ services:
## Lbrycrd ##
#############
lbrycrd:
image: tiger5226/testnet
image: lbry/lbrycrd:linux-x86_64-production
restart: always
labels:
- "traefik.expose=true"
ports:
- "11336:9246"
- "11337:11337"
## host volumes for persistent data such as wallet private keys.
volumes:
- "./data:/data"
- "../persist/data:/data"
environment:
- RUN_MODE=testnet
-41
View File
@@ -1,41 +0,0 @@
## This base image is for running latest lbrycrdd
# For some reason I may switch this image over to Alpine when I can RCA why it won't start.
FROM ubuntu:18.04
LABEL MAINTAINER="beamer@lbry.io"
RUN addgroup --gid 1000 lbrycrd && \
adduser lbrycrd --uid 1000 --gid 1000 --gecos GECOS --shell /bin/bash --disabled-password --home /data && \
apt-get update && \
apt-get -y install unzip wget curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/*
## TODO: Consider adding debugpaste or variant
# RUN wget -O /usr/bin/debugpaste https://github.com/nixc-us/debugpaste-it/raw/master/bin/debugpaste_64 && \
# chmod +x /usr/bin/debugpaste
RUN wget -O /usr/bin/lbrycrd-linux.zip https://github.com/lbryio/lbrycrd/releases/download/v0.12.4.0/lbrycrd-linux.zip && \
cd /usr/bin/ && \
unzip lbrycrd-linux.zip && \
rm lbrycrd-linux.zip && \
chmod +x lbrycrdd lbrycrd-cli lbrycrd-tx
COPY start.sh /usr/local/bin/start
COPY advance_blocks.sh /usr/local/bin/advance
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
# USER lbrycrd
# RUN mkdir /data
VOLUME ["/data"]
WORKDIR /data
## Exposing daemon port and RPC port
EXPOSE 9245 9246
## TODO: Decide what's important for lbrycrd and possibly add an entrypoint.
## Maybe catch things that might match things that can be easily executed in the
## lbrycrd cli and if nothing is entered just default to the containers shell.
## For now this is a placeholder that executes /bin/bash on `docker exec`
# ENTRYPOINT ["docker-entrypoint"]
CMD ["start"]
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-34
View File
@@ -1,34 +0,0 @@
#!/usr/bin/env bash
# ## ToDo: Get a test case to see if this is the first run or a repeat run.
# ## If it's a first run you need to do a full index including all transactions
# ## tx index creates an index of every single transaction in the block history if
# ## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
# ## This is specific to chainquery.
## Ensure perms are correct prior to running main binary
mkdir -p /data/.lbrycrd
chown -R lbrycrd:lbrycrd /data
chmod -R 755 /data/
## TODO: Consider a config directory for future magic.
# chown -R 1000:1000 /etc/lbrycrd
# chmod -R 755 /etc/lbrycrd
rm -f /var/run/lbrycrd.pid
## Set config params
## TODO: Make this more automagic in the future.
echo "rpcuser=lbry" > /data/.lbrycrd/lbrycrd.conf
echo "rpcpassword=lbry" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcport=11337" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcbind=0.0.0.0" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcallowip=0.0.0.0/0" >> /data/.lbrycrd/lbrycrd.conf
echo "regtest=1" >> /data/.lbrycrd/lbrycrd.conf
echo "txindex=1" >> /data/.lbrycrd/lbrycrd.conf
echo "server=1" >> /data/.lbrycrd/lbrycrd.conf
echo "printtoconsole=1" >> /data/.lbrycrd/lbrycrd.conf
#nohup advance &>/dev/null &
su -c "lbrycrdd -conf=/data/.lbrycrd/lbrycrd.conf" lbrycrd
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
docker build -t tiger5226/regtest:latest .
docker push tiger5226/regtest:latest
@@ -2,4 +2,4 @@
while true; do
lbrycrd-cli -conf=/data/.lbrycrd/lbrycrd.conf generate 1 >> /tmp/output.log
sleep 2
done
done
-3
View File
@@ -1,3 +0,0 @@
#!/usr/bin/env bash
echo "placeholder for now, come back soon for more debugging magic"
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
+66 -22
View File
@@ -1,20 +1,23 @@
#!/usr/bin/env bash
CONFIG_PATH=/etc/lbry/lbrycrd.conf
if [ -f "$CONFIG_PATH" ]
then
echo "Using the config file that was mounted into the container."
else
echo "Creating a fresh config file from environment variables."
## Set config params
mkdir -p `dirname $CONFIG_PATH`
echo "rpcuser=$RPC_USER" > $CONFIG_PATH
echo "rpcpassword=$RPC_PASSWORD" >> $CONFIG_PATH
echo "rpcallowip=$RPC_ALLOW_IP" >> $CONFIG_PATH
echo "rpcport=9245" >> $CONFIG_PATH
echo "rpcbind=0.0.0.0" >> $CONFIG_PATH
#echo "bind=0.0.0.0" >> $CONFIG_PATH
fi
function set_config() {
CONFIG_PATH=/etc/lbry/lbrycrd.conf
if [ -f "$CONFIG_PATH" ]
then
echo "Using the config file that was mounted into the container."
else
echo "Creating a fresh config file from environment variables."
## Set config params
echo "rpcuser=$RPC_USER" > $CONFIG_PATH
echo "rpcpassword=$RPC_PASSWORD" >> $CONFIG_PATH
echo "rpcallowip=$RPC_ALLOW_IP" >> $CONFIG_PATH
echo "rpcport=9245" >> $CONFIG_PATH
echo "rpcbind=0.0.0.0" >> $CONFIG_PATH
#echo "bind=0.0.0.0" >> $CONFIG_PATH
fi
}
## Ensure perms are correct prior to running main binary
/usr/bin/fix-permissions
@@ -22,19 +25,60 @@ fi
## You can optionally specify a run mode if you want to use lbry defined presets for compatibility.
case $RUN_MODE in
default )
set_config
lbrycrdd -server -conf=$CONFIG_PATH -printtoconsole
;;
## If it's a first run you need to do a full index including all transactions
## tx index creates an index of every single transaction in the block history if
## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
## This is generally specific to chainquery.
## If it's a first run you need to do a full index including all transactions
## tx index creates an index of every single transaction in the block history if
## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
## This is generally specific to chainquery.
reindex )
## Apply this RUN_MODE in the case you need to update a dataset. NOTE: you do not need to use `RUN_MODE reindex` for more than one complete run.
## Apply this RUN_MODE in the case you need to update a dataset. NOTE: you do not need to use `RUN_MODE reindex` for more than one complete run.
set_config
lbrycrdd -server -txindex -reindex -conf=$CONFIG_PATH -printtoconsole
;;
chainquery )
## If your only goal is to run Chainquery against this instance of lbrycrd and you're starting a
## fresh local dataset use this run mode.
## If your only goal is to run Chainquery against this instance of lbrycrd and you're starting a
## fresh local dataset use this run mode.
set_config
lbrycrdd -server -txindex -conf=$CONFIG_PATH -printtoconsole
;;
regtest )
## Set config params
## TODO: Make this more automagic in the future.
mkdir -p `dirname $CONFIG_PATH`
echo "rpcuser=lbry" > $CONFIG_PATH
echo "rpcpassword=lbry" >> $CONFIG_PATH
echo "rpcport=11337" >> $CONFIG_PATH
echo "rpcbind=0.0.0.0" >> $CONFIG_PATH
echo "rpcallowip=0.0.0.0/0" >> $CONFIG_PATH
echo "regtest=1" >> $CONFIG_PATH
echo "txindex=1" >> $CONFIG_PATH
echo "server=1" >> $CONFIG_PATH
echo "printtoconsole=1" >> $CONFIG_PATH
#nohup advance &>/dev/null &
lbrycrdd -conf=$CONFIG_PATH $1
;;
testnet )
## Set config params
## TODO: Make this more automagic in the future.
mkdir -p `dirname $CONFIG_PATH`
echo "rpcuser=lbry" > $CONFIG_PATH
echo "rpcpassword=lbry" >> $CONFIG_PATH
echo "rpcport=11337" >> $CONFIG_PATH
echo "rpcbind=0.0.0.0" >> $CONFIG_PATH
echo "rpcallowip=0.0.0.0/0" >> $CONFIG_PATH
echo "testnet=1" >> $CONFIG_PATH
echo "txindex=1" >> $CONFIG_PATH
echo "server=1" >> $CONFIG_PATH
echo "printtoconsole=1" >> $CONFIG_PATH
#nohup advance &>/dev/null &
lbrycrdd -conf=$CONFIG_PATH $1
;;
* )
echo "Error, you must define a RUN_MODE environment variable."
echo "Available options are testnet, regtest, chainquery, default, and reindex"
;;
esac
-40
View File
@@ -1,40 +0,0 @@
## This base image is for running latest lbrycrdd
# For some reason I may switch this image over to Alpine when I can RCA why it won't start.
FROM ubuntu:18.04
LABEL MAINTAINER="leopere [at] nixc [dot] us"
RUN addgroup --gid 1000 lbrycrd && \
adduser lbrycrd --uid 1000 --gid 1000 --gecos GECOS --shell /bin/bash --disabled-password --home /data && \
apt-get update && \
apt-get -y install unzip wget curl && \
apt-get autoclean -y && \
rm -rf /var/lib/apt/lists/*
## TODO: Consider adding debugpaste or variant
# RUN wget -O /usr/bin/debugpaste https://github.com/nixc-us/debugpaste-it/raw/master/bin/debugpaste_64 && \
# chmod +x /usr/bin/debugpaste
RUN wget -O /usr/bin/lbrycrd-linux.zip https://github.com/lbryio/lbrycrd/releases/download/v0.12.3.1/lbrycrd-linux.zip && \
cd /usr/bin/ && \
unzip lbrycrd-linux.zip && \
rm lbrycrd-linux.zip && \
chmod +x lbrycrdd lbrycrd-cli lbrycrd-tx
COPY start.sh /usr/local/bin/start
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint
# USER lbrycrd
# RUN mkdir /data
VOLUME ["/data"]
WORKDIR /data
## Exposing daemon port and RPC port
EXPOSE 9245 9246
## TODO: Decide what's important for lbrycrd and possibly add an entrypoint.
## Maybe catch things that might match things that can be easily executed in the
## lbrycrd cli and if nothing is entered just default to the containers shell.
## For now this is a placeholder that executes /bin/bash on `docker exec`
# ENTRYPOINT ["docker-entrypoint"]
CMD ["start"]
-7
View File
@@ -1,7 +0,0 @@
To run a local instance of testnet via docker:
1) clone this repository
2) Run `cd ./lbrycrd/testnet`
3) Run `docker-compose up -d`
-13
View File
@@ -1,13 +0,0 @@
#!/usr/bin/env bash
# default to run whatever the user wanted like "/bin/bash"
## If user runs no need to run any more of the entrypoint script.
if [[ -z "$@" ]]; then
echo "User did not attempt input. Now executing docker-entrypoint."
else
echo "Running $@."
exec "$@"
exit 1
fi
/bin/bash
-34
View File
@@ -1,34 +0,0 @@
#!/usr/bin/env bash
# ## ToDo: Get a test case to see if this is the first run or a repeat run.
# ## If it's a first run you need to do a full index including all transactions
# ## tx index creates an index of every single transaction in the block history if
# ## not specified it will only create an index for transactions that are related to the wallet or have unspent outputs.
# ## This is specific to chainquery.
## Ensure perms are correct prior to running main binary
mkdir -p /data/.lbrycrd
chown -R lbrycrd:lbrycrd /data
chmod -R 755 /data/
## TODO: Consider a config directory for future magic.
# chown -R 1000:1000 /etc/lbrycrd
# chmod -R 755 /etc/lbrycrd
rm -f /var/run/lbrycrd.pid
## Set config params
## TODO: Make this more automagic in the future.
echo "rpcuser=lbry" > /data/.lbrycrd/lbrycrd.conf
echo "rpcpassword=lbry" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcport=11337" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcbind=0.0.0.0" >> /data/.lbrycrd/lbrycrd.conf
echo "rpcallowip=0.0.0.0/0" >> /data/.lbrycrd/lbrycrd.conf
echo "testnet=1" >> /data/.lbrycrd/lbrycrd.conf
echo "txindex=1" >> /data/.lbrycrd/lbrycrd.conf
echo "server=1" >> /data/.lbrycrd/lbrycrd.conf
echo "printtoconsole=1" >> /data/.lbrycrd/lbrycrd.conf
#nohup advance &>/dev/null &
su -c "lbrycrdd -conf=/data/.lbrycrd/lbrycrd.conf" lbrycrd
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
docker build -t tiger5226/testnet:latest .
docker push tiger5226/testnet:latest
-1
View File
@@ -1 +0,0 @@
data/
@@ -0,0 +1,68 @@
## Compiler for lbrynet container for any architecture supported by Ubuntu 18.04
## Specify the BASE_IMAGE build argument to choose which Ubuntu base image to build from.
## Docs for ARG in FROM: https://github.com/docker/cli/blob/master/docs/reference/builder.md#understand-how-arg-and-from-interact
## Architecture | Build command
## x86_64 | docker build -t lbrynet -f Dockerfile-compiler .
## armhf | docker build -t lbrynet-armhf -f Dockerfile-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:armhf-bionic .
## arm64 | docker build -t lbrynet-arm64 -f Dockerfile-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:arm64-bionic .
ARG BASE_IMAGE=ubuntu:18.04
FROM ${BASE_IMAGE} as dependencies
RUN apt-get update && \
apt-get install -y \
python3-pip \
python3.7 \
python3.7-dev \
build-essential \
libssl-dev \
libacl1-dev \
liblz4-dev \
libfuse-dev \
fuse \
pkg-config \
fakeroot \
git \
zlib1g-dev \
libbz2-dev \
libncurses5-dev \
libreadline-dev \
liblzma-dev \
libsqlite3-dev \
zip \
libffi-dev \
libleveldb-dev && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1 && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 10 && \
update-alternatives --config python && \
python --version && \
pip3 --version
FROM dependencies as compile
ARG VERSION=master
RUN python3.7 -m pip install -U pyinstaller && \
git clone https://github.com/lbryio/lbry.git /lbry && \
git -C /lbry checkout ${VERSION}
WORKDIR /lbry
RUN git clone https://github.com/lbryio/torba.git --depth 1 /lbry/torba
WORKDIR /lbry/torba
RUN python3.7 -m pip install -e .
WORKDIR /lbry/
COPY stuff/start.sh /usr/local/bin/start
COPY stuff/checkmount.sh /usr/local/bin/checkmount
RUN python3.7 scripts/set_build.py && \
python3.7 -m pip install -e . && \
pyinstaller -F -n lbrynet lbrynet/extras/cli.py && \
chmod +x /lbry/dist/lbrynet && \
zip -j /lbry/dist/lbrynet-armhf.zip /lbry/dist/lbrynet && \
chmod a+x /usr/local/bin/* && \
mkdir /target && \
/lbry/dist/lbrynet --version
FROM ${BASE_IMAGE} as app
RUN adduser lbrynet --gecos GECOS --shell /bin/bash --disabled-password --home /home/lbrynet
COPY --from=compile /usr/local/bin/start /usr/local/bin/checkmount /lbry/dist/lbrynet /usr/local/bin/
EXPOSE 5279
USER lbrynet
ENTRYPOINT ["/usr/local/bin/checkmount"]
CMD ["start"]
@@ -2,20 +2,18 @@
FROM ubuntu:18.04
LABEL MAINTAINER="leopere [at] nixc [dot] us"
RUN apt-get update && apt-get -y install unzip
RUN adduser lbrynet --gecos GECOS --shell /bin/bash --disabled-password --home /lbrynet/
## Add lbrynet
ADD https://lbry.io/get/lbrynet.linux.zip /lbrynet/lbrynet.linux.zip
RUN unzip /lbrynet/lbrynet.linux.zip -d /lbrynet/ && \
RUN apt-get update && apt-get -y install unzip && \
adduser lbrynet --gecos GECOS --shell /bin/bash --disabled-password --home /home/lbrynet && \
unzip /lbrynet/lbrynet.linux.zip -d /lbrynet/ && \
rm /lbrynet/lbrynet.linux.zip && \
chown -Rv lbrynet:lbrynet /lbrynet && \
mv /lbrynet/lbrynet /bin/
# COPY stuff/debugpaste-it.sh /usr/local/bin/debugpaste-it
COPY stuff/start.sh /usr/local/bin/start
COPY stuff/docker-entrypoint.sh /usr/local/bin/docker-entrypoint
COPY stuff/healthcheck.sh /usr/local/bin/healthcheck
COPY stuff/checkmount.sh /usr/local/bin/checkmount
## Daemon port [Intended for internal use]
## LBRYNET talks to peers on port 3333 [Intended for external use] this port is used to discover other lbrynet daemons with blobs.
@@ -24,21 +22,9 @@ COPY stuff/healthcheck.sh /usr/local/bin/healthcheck
## the lbryumx aka Wallet port [Intended for internal use]
EXPOSE 4444 3333 5566 5279 50001
## TODO: Have wallets stored on their own volume/subdirectory for security, and backups.
## Volumize the wallets in a separate location for backup purposes may be unnecessary.
# VOLUME /lbrynet/.local/share/lbry/lbryum/wallets
# VOLUME /lbrynet/.local/
## Example daemon_settings.yml is at https://github.com/lbryio/lbry/blob/master/example_daemon_settings.yml
ADD https://raw.githubusercontent.com/lbryio/lbry/master/example_daemon_settings.yml /etc/lbrynet/example_daemon_settings.yml
## TODO: In start.sh detect and merge changes.
# VOLUME /etc/lbry/daemon_settings.yml
## TODO: Downloaded blobs will be in their own separate volume for keeping backups of critical secrets and data separate from backups of potentially massive blob files.
# VOLUME /lbrynet/Downloads/
ENTRYPOINT ["/bin/bash"]
EXPOSE 5279
USER lbrynet
ENTRYPOINT ["/usr/local/bin/checkmount"]
CMD ["start"]
+51 -14
View File
@@ -1,24 +1,61 @@
## lbrynet-As-A-Container
# lbrynet
# Docker image tags
`lbry/lbrycrd`
`[linux-x86_64-production](https://github.com/lbryio/lbry-docker/blob/master/chainquery/Dockerfile-linux-x86_64-production)` (Latest release)
More documentation to come however the Invocation section should include enough to get you going. You're going to need at least docker docker-compose and git installed on whatever host OS you wish to use.
## Compiler container
The [Dockerfile-linux-multiarch-compiler](Dockerfile-linux-multiarch-compiler) is for building lbrynet for any architecture supported
by an Ubuntu 18.04 base image.
### Register qemu to run docker images built for platforms other than your host
#### Invocation
This will get you a running copy of the lbrynet-daemon running inside of a docker container with default settings.
```
git clone https://github.com/chamunks/lbry-docker.git
cd ./lbry-docker/lbrynet/
docker-compose up -d
docker run --rm --privileged multiarch/qemu-user-static:register
```
#### Executing commands
### Build for the default x86_64 platform:
To list containers on the host execute `docker ps -a` then run `docker exec CONTAINERNAME lbrynet-cli commands`
```
docker build -t lbrynet -f Dockerfile-linux-multiarch-compiler .
```
#### Docker Directory
### Build for an ARM 32-bit platform:
This directory is in case we need to expand the functionality of this container at some point in the future.
```
docker build -t lbrynet-armhf -f Dockerfile-linux-multiarch-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:armhf-bionic .
```
#### Configuration
There's really no configuration required to launch this just launch it. However your blockchain data and other things are currently located in the applications home Directory here's a link to the [Documentation](https://lbry.io/faq/lbry-directories) for useful directories with lbrynet-daemon
### Build for an ARM 64-bit platform:
*daemon_settings.yml* is on its way and it will be configurable soon via env-vars with *docker-compose.yml*
```
docker build -t lbrynet-arm64 -f Dockerfile-linux-multiarch-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:arm64-bionic .
```
### Extra build arguments
#### VERSION
Compile any version of lbrynet by specifying the git tag:
```
docker build -t lbrynet:v0.36.0 --build-arg VERSION=v0.36.0 -f Dockerfile-linux-multiarch-compiler .
```
### Running from the compiler container directly
The container requires a home directory to be mounted at `/home/lbrynet`. This
is to ensure that the wallet is backed up to a real storage device. You must run
the container with the appropriate volume argument, or else lbrynet will refuse
to run.
If you compiled lbrynet as above, with the tag `lbrynet-x86`, you could run
docker like so:
```
docker run --rm -it -v wallet:/home/lbrynet lbrynet-x86 lbrynet start
```
This automatically creates a docker volume called `wallet` and it will persist
across container restarts. See more in the [Docker volume
documentation](https://docs.docker.com/storage/volumes/)
-3
View File
@@ -1,3 +0,0 @@
## Planned container [WIP/Planned]
This will eventually be a container for compiling the lbrynet-daemon from source.
Anyone who wants to replicate the build process in case they feel like wearing a tin foil hat or contributing to the project could use this.
@@ -9,19 +9,18 @@ services:
## Lbrynet ##
#############
lbrynet:
build: .
image: lbry/lbrynet:Linux-x86_64-production
restart: always
## TODO: Does this container ever need a static network address or is it better to be dynamic?
networks:
# lbrynet:
# ipv4_address: 10.6.1.4
- lbry-network
labels:
- "traefik.expose=false"
## TODO: Verify the need for the following port binds on host networking.
ports:
- 4444:4444
- 50001:50001
env_file:
- ../environment/lbrynet.env
## host volumes for persistent data such as wallet private keys.
# This container will not run without a persistent wallet as it may contain a LBC balance.
volumes:
- ./data:/lbrynet
- ../persist/lbrynet/data:/lbrynet
-52
View File
@@ -1,52 +0,0 @@
FROM multiarch/ubuntu-core:armhf-bionic as dependencies
RUN apt-get update && \
apt-get install -y \
python3-pip \
python3.7 \
python3.7-dev \
build-essential \
libssl-dev \
libacl1-dev \
liblz4-dev \
libfuse-dev \
fuse \
pkg-config \
fakeroot \
git \
zlib1g-dev \
libbz2-dev \
libncurses5-dev \
libreadline-dev \
liblzma-dev \
libsqlite3-dev \
zip \
libffi-dev \
libleveldb-dev && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1 && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 10 && \
update-alternatives --config python && \
python --version && \
pip3 --version
FROM dependencies as compile
RUN python3.7 -m pip install -U pyinstaller && \
git clone https://github.com/lbryio/lbry.git --depth 1 /lbry && \
cd /lbry && \
git clone https://github.com/lbryio/torba.git --depth 1 /lbry/torba && \
cd /lbry/torba && python3.7 -m pip install -e . && \
cd /lbry/ && python3.7 scripts/set_build.py && \
python3.7 -m pip install -e . && \
pyinstaller -F -n lbrynet lbrynet/extras/cli.py && \
chmod +x /lbry/dist/lbrynet && \
zip -j /lbry/dist/lbrynet-armhf.zip /lbry/dist/lbrynet && \
mkdir /target && \
/lbry/dist/lbrynet --version
FROM multiarch/ubuntu-core:armhf-bionic as app
RUN adduser lbrynet --gecos GECOS --shell /bin/bash --disabled-password --home /home/lbrynet
COPY start.sh /usr/local/bin/start
COPY --from=compile /lbry/dist/lbrynet /usr/local/bin/
EXPOSE 5279
USER lbrynet
CMD ["start"]
-28
View File
@@ -1,28 +0,0 @@
# armhf-compiler
This container's goal is to make CI/CD easier for everyone, Travis CI, GitlabCI, Jenkins... Your desktop's docker equipped development environment.
## Example Usage
#### binfmt_misc register
This step sets up your docker daemon to support more container architectures.
Register `quemu-*-static` for all supported processors except the current one.
* `docker run --rm --privileged multiarch/qemu-user-static:register`
#### build the armhf bin
<!-- TODO: Process could be greatly sped up but keeping it simple for first release. -->
* `docker build --tag lbryio/lbrynet:armhf-compiler .`
#### export compiled bin to local /target
This containers sole purpose is to build and spit out the bin.
<!-- TODO: Fork this container base to begin work on LbryTV compiler to reduce build time on rpi -->
* `docker run --rm -ti -v $(pwd)/target:/target lbryio/lbrynet:armhf-compiler`
## Cleanup
If you're doing this on a machine you care to have restored to defaults this is the only host change we imposed so to revert the change you must execute the following docker command.
Same as above, but remove all registered `binfmt_misc` before
* `docker run --rm --privileged multiarch/qemu-user-static:register --reset`
-2
View File
@@ -1,2 +0,0 @@
#!/bin/bash
echo "TODO"
-52
View File
@@ -1,52 +0,0 @@
FROM ubuntu:18.04 as dependencies
RUN apt-get update && \
apt-get install -y \
python3-pip \
python3.7 \
python3.7-dev \
build-essential \
libssl-dev \
libacl1-dev \
liblz4-dev \
libfuse-dev \
fuse \
pkg-config \
fakeroot \
git \
zlib1g-dev \
libbz2-dev \
libncurses5-dev \
libreadline-dev \
liblzma-dev \
libsqlite3-dev \
zip \
libffi-dev \
libleveldb-dev && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1 && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 10 && \
update-alternatives --config python && \
python --version && \
pip3 --version
FROM dependencies as compile
RUN python3.7 -m pip install -U pyinstaller && \
git clone https://github.com/lbryio/lbry.git --depth 1 /lbry && \
cd /lbry && \
git clone https://github.com/lbryio/torba.git --depth 1 /lbry/torba && \
cd /lbry/torba && python3.7 -m pip install -e . && \
cd /lbry/ && python3.7 scripts/set_build.py && \
python3.7 -m pip install -e . && \
pyinstaller -F -n lbrynet lbrynet/extras/cli.py && \
chmod +x /lbry/dist/lbrynet && \
zip -j /lbry/dist/lbrynet-armhf.zip /lbry/dist/lbrynet && \
mkdir /target && \
/lbry/dist/lbrynet --version
FROM ubuntu:18.04 as app
RUN adduser lbrynet --gecos GECOS --shell /bin/bash --disabled-password --home /home/lbrynet
COPY start.sh /usr/local/bin/start
COPY --from=compile /lbry/dist/lbrynet /usr/local/bin/
EXPOSE 5279
USER lbrynet
CMD ["start"]
-12
View File
@@ -1,12 +0,0 @@
# x86_64-compiler
This container's goal is to make CI/CD easier for everyone, Travis CI, GitlabCI, Jenkins... Your desktop's docker equipped development environment.
## Example Usage
#### build the x86_64 bin
* `docker build --tag lbryio/lbrynet:x86_64-compiler .`
#### export compiled bin to local /target
This containers sole purpose is to build and spit out the x86_64 binary.
* `docker run --rm -ti -v $(pwd)/target:/target lbryio/lbrynet:x86_64-compiler`
-2
View File
@@ -1,2 +0,0 @@
#!/bin/bash
echo "TODO"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/bash
## TODO: Make a bit more aware of the run mode of this appliance in case there is ever a test mode enabled in the start.sh
mountpoint=/home/lbrynet
if ! grep -qs ".* $mountpoint " /proc/mounts; then
echo "$mountpoint not mounted, refusing to run."
## TODO: We should have documentation that this error references directly with a URL as to why it won't run without a volume.
exit 1
else
`$@`
fi
-2
View File
@@ -1,2 +0,0 @@
#!/usr/bin/env bash
echo "For now we're going to go with a tune in next time for a debugpaste."
-3
View File
@@ -1,3 +0,0 @@
#!/usr/bin/env bash
## TODO: Start work for the docker-entrypoint.sh for lbrynet daemon
-2
View File
@@ -1,2 +0,0 @@
#!/usr/bin/env bash
## TODO: write a healthcheck script.
+2 -17
View File
@@ -1,17 +1,2 @@
#!/usr/bin/env bash
## Scope: This start.sh script will asssert container filesystem permissions and
## then execute the desired run mode for lbrynet with reduced permissions.
## The other thing this should do is simply create, configure or simply establish
## a fresh config with envvars passed to the container.
## Ensure perms are correct prior to running main binary
mkdir -p /lbrynet
chown -R lbrynet:lbrynet /lbrynet
chmod -R 755 /lbrynet
## TODO: Consider a config directory for future magic.
# chown -R 1000:1000 /etc/lbrynet
# chmod -R 755 /etc/lbrynet
# rm -f /var/run/lbrynet.pid
su -c "lbrynet start" lbrynet
#!/bin/bash
lbrynet start
File renamed without changes.