Compare commits

...
1496 Commits
Author SHA1 Message Date
Lex Berezhny e01fbc6f39 initial import of android lbrysdk build 2019-01-27 22:45:32 -05:00
Lex Berezhny 60ed4e503f refactored settings_set to support all settings 2019-01-26 00:04:36 -05:00
Lex Berezhny aa1405887e use float instead of decimal for now 2019-01-25 23:20:43 -05:00
Lex Berezhny cced217e43 update doc format for max_key_fee 2019-01-25 23:20:43 -05:00
Lex Berezhny 8a3b8d2df1 max_key_fee in yaml as null 2019-01-25 23:20:43 -05:00
Lex Berezhny be0bd3bdea refactored --max-key-fee to be more ergonomic 2019-01-25 23:20:43 -05:00
Lex Berezhny e01c73a7f8 pylint 2019-01-25 21:00:20 -05:00
Lex Berezhny 134338db4b dropped help, commands and resolve_name commands 2019-01-25 21:00:20 -05:00
Lex Berezhny f47a685734 help message for start command 2019-01-25 20:28:47 -05:00
Lex Berezhny 6e29f7e99b user is shown help when typing lbrynet help 2019-01-25 20:28:47 -05:00
Lex Berezhny 026d93e92a Update .travis.yml
use `stage: build` for the docker builds to get some parallelism.
2019-01-25 17:28:06 -05:00
Lex Berezhny a55802c1de v0.31.0rc3 2019-01-25 17:05:38 -05:00
Jack Robison 894397a327 Merge pull request #1810 from lbryio/async-reflector
async reflector
2019-01-25 16:41:37 -05:00
Jack Robison 3e3f6c0e22 fix test 2019-01-25 16:20:19 -05:00
Jack Robison a7610e3d34 async reflector 2019-01-25 16:10:17 -05:00
Jack Robison d0e6e74e1b Merge pull request #1809 from lbryio/fix-blob-server
fix blob server protocol factory
2019-01-25 16:07:07 -05:00
Jack Robison 126d71978b disable slow test 2019-01-25 15:38:38 -05:00
Jack Robison 69bcb052de fix blob server protocol factory 2019-01-25 13:10:40 -05:00
Victor Shyba 071af12d1a release wallet server master tag every master merge 2019-01-25 12:20:42 -05:00
Lex Berezhny b7d62ca084 v0.31.0rc2 2019-01-25 10:46:20 -05:00
Jack Robison fa97f7bd38 fix written_bytes error 2019-01-25 10:23:03 -05:00
Lex Berezhny 4a35247016 fixes for toggle config, passing servers as list, etc 2019-01-25 10:19:24 -05:00
Lex Berezhny d0526ad7d5 bring --config back to start command 2019-01-25 10:19:24 -05:00
Lex Berezhny 19b529cf38 removed unused import 2019-01-25 10:19:24 -05:00
Lex Berezhny 143353be44 fix cli unit test 2019-01-25 10:19:24 -05:00
Lex Berezhny 0373196707 max_key_fee is plain YAML now 2019-01-25 10:19:24 -05:00
Lex Berezhny 7a601db23f cli fixup 2019-01-25 10:19:24 -05:00
Victor Shyba ed61c7feec add step to build wallet server images on release 2019-01-24 19:20:23 -05:00
Jack Robison 77ece96356 fix config error if $HOME is None
-not seen on travis due to HOME being manually set to /tmp
2019-01-24 12:42:55 -05:00
Lex Berezhny 80440fd6bd v0.31.0rc1 2019-01-24 00:13:55 -05:00
Lex Berezhny ed6537db79 integration test fixes 2019-01-24 00:13:15 -05:00
Lex Berezhny 79e4767f69 unused imports 2019-01-24 00:13:15 -05:00
Lex Berezhny 0ee279bfd7 fix startup/shutdown and file_list --full-status argument 2019-01-24 00:13:15 -05:00
Lex Berezhny 5249d2a3ca removed unused methods 2019-01-24 00:13:15 -05:00
Lex Berezhny 4d5823f473 distro package is in setup.py, does not make sense to wrap it in try/except ModuleNotFoundError 2019-01-24 00:13:15 -05:00
Lex Berezhny c2b9909397 moved tests/unit/lbrynet_daemon/test_docs.py into tests/unit/test_cli.py 2019-01-24 00:13:15 -05:00
Lex Berezhny afff362544 component tests need AsyncioTestCase 2019-01-24 00:13:15 -05:00
Lex Berezhny 64eb7dbc49 renamed test_Component_Manager.py to test_component_manager.py 2019-01-24 00:13:15 -05:00
Lex Berezhny 2dcd187d69 fixed component manager tests 2019-01-24 00:13:15 -05:00
Lex Berezhny 3d9ea92118 remove unnecessary logger configuration for BitcoinRPC 2019-01-24 00:13:15 -05:00
Lex Berezhny 13a18d422e dropped envparse 2019-01-24 00:13:15 -05:00
Lex Berezhny 1bbbbd289c TestCase -> AsyncioTestCase 2019-01-24 00:13:15 -05:00
Lex Berezhny e33b74f60c import mock from unittest m0ar 2019-01-24 00:13:15 -05:00
Lex Berezhny cd76e0c296 import mock from unittest 2019-01-24 00:13:15 -05:00
Lex Berezhny 3b61222e5a pylint 2019-01-24 00:13:15 -05:00
Lex Berezhny 2ca438c36e passing integration tests 2019-01-24 00:13:15 -05:00
Jack Robison 8948d5be86 don't shut down immediately after startup 2019-01-24 00:13:15 -05:00
Jack Robison a3e64f9cef remove dht requirement from stream manager component 2019-01-24 00:13:15 -05:00
Jack Robison 09afec1961 import 2019-01-24 00:13:15 -05:00
Jack Robison f9c465a636 f string 2019-01-24 00:13:15 -05:00
Jack Robison 89efe3d1b9 minimal changes to cli.py 2019-01-24 00:13:15 -05:00
Jack Robison 94cec1b86d fix 2019-01-24 00:13:15 -05:00
Jack Robison d62ac0b608 KeyboardInterrupt 2019-01-24 00:13:15 -05:00
Jack Robison 5596eac930 merge conflicts 2019-01-24 00:13:15 -05:00
Jack Robison 59f5381b0a update tests 2019-01-24 00:13:15 -05:00
Jack Robison 65ae1b3f08 update setup.py, travis, and wine_build 2019-01-24 00:13:15 -05:00
Jack Robison 977a7b34f4 update pylintrc 2019-01-24 00:13:15 -05:00
Jack Robison c01728a6a7 update and add scripts 2019-01-24 00:13:15 -05:00
Jack Robison 5bb94d744e storage 2019-01-24 00:13:15 -05:00
Jack Robison 494917158c update daemon and components 2019-01-24 00:13:15 -05:00
Jack Robison ea15674e62 delete old dht functional tests 2019-01-24 00:13:15 -05:00
Jack Robison 34a409ade5 delete log_support, looping_call_manager, and compat 2019-01-24 00:13:15 -05:00
Jack Robison 273e614715 delete old reflector code 2019-01-24 00:13:15 -05:00
Jack Robison 34a725f7ad fix conf 2019-01-24 00:13:15 -05:00
Jack Robison bb6112276f delete old tests 2019-01-24 00:13:15 -05:00
Jack Robison db75f0baef update SQLiteStorage 2019-01-24 00:13:15 -05:00
Jack Robison 30689bbf8d update/add config
remove the following settings:
  peer_search_timeout
  sd_download_timeout
  run_reflector_server
  download_mirrors
  pointtrader_server

add the following settings:
  blob_download_timeout
  peer_connect_timeout
  node_rpc_timeout
2019-01-24 00:13:15 -05:00
Jack Robison 4bf7742c19 async lbrynet.stream 2019-01-24 00:13:15 -05:00
Jack Robison 69b259c285 async lbrynet.blob_exchange 2019-01-24 00:13:15 -05:00
Jack Robison 2fa5233796 async lbrynet.dht 2019-01-24 00:13:15 -05:00
Jack Robison a5524d490c async lbrynet.blob 2019-01-24 00:13:15 -05:00
Jack Robison c713fac2d9 lbrynet.error 2019-01-24 00:13:15 -05:00
Lex Berezhny 7a038bbb98 cli bug fixes 2019-01-23 13:25:10 -05:00
Victor Shyba eba88c1df7 fix resolve for undecodeable claims 2019-01-23 13:22:23 -05:00
Victor Shyba 881e5aba68 simplify claim_list 2019-01-23 13:22:23 -05:00
Victor Shyba 368f0683ec fix claim_list for channels 2019-01-23 09:36:27 -05:00
Victor Shyba 94dc8c8520 add tests for claim_list 2019-01-23 09:36:27 -05:00
Lex Berezhny 8593cd362a check darwin before windows because darwin has win, face plam! 2019-01-22 18:04:17 -05:00
Lex Berezhny 2dbf96da81 blob downer test fixes 2019-01-22 18:04:17 -05:00
Lex Berezhny 83c7797187 pylint 2019-01-22 18:04:17 -05:00
Lex Berezhny 6606deafa5 fixed after testing with desktop app 2019-01-22 18:04:17 -05:00
Lex Berezhny 19b0d59159 config options api_host and api_port changed to just api 2019-01-22 18:04:17 -05:00
Lex Berezhny 9ccb3fa2a3 unit/test_cli tests 2019-01-22 18:04:17 -05:00
Lex Berezhny 48533df523 fix integration tests 2019-01-22 18:04:17 -05:00
Lex Berezhny 1ae1137005 fixing integration tests 2019-01-22 18:04:17 -05:00
Lex Berezhny d5f2832b2f pylint 2019-01-22 18:04:17 -05:00
Lex Berezhny d0230b4893 command line fixes 2019-01-22 18:04:17 -05:00
Lex Berezhny a6cd53b97c test_Daemon.py uses /tmp for data_dir 2019-01-22 18:04:17 -05:00
Lex Berezhny 7a28171a72 conf no longer global 2019-01-22 18:04:17 -05:00
Lex Berezhny 150bcb1116 new configuration manager, first pass 2019-01-22 18:04:17 -05:00
Jack Robison 6fe9c5bf00 Merge pull request #1794 from lbryio/async-exchange-rate-manager
refactor exchange rate manager to use asyncio
2019-01-22 12:05:07 -05:00
Jack Robison 3a86fc6680 refactor exchange rate manager to use asyncio 2019-01-22 11:34:49 -05:00
Jack Robison 2805cbb331 refactor loggly handler to use asyncio 2019-01-22 10:52:50 -05:00
Jack Robison 68d7a7014c Merge pull request #1792 from lbryio/delete-unused-scripts
delete unused scripts
2019-01-22 09:53:24 -05:00
Jack Robison 883fa267a5 delete unused scripts 2019-01-22 09:33:42 -05:00
Lex Berezhny 5f2c3ac965 dont run customlogger tests 2019-01-19 16:32:11 -05:00
Lex Berezhny bf8e32ebd2 remove custom logging 2019-01-19 16:30:19 -05:00
Lex Berezhny 5971be5bef v0.30.5 2019-01-18 16:14:48 -05:00
Lex Berezhny 677d76a3e4 v0.30.5rc4 2019-01-17 23:26:12 -05:00
Victor Shyba 6ed3207cb7 phase1: legacy claim format, ability to decode new format 2019-01-17 23:24:58 -05:00
Victor Shyba be6448b74f fix str/bytes confusion from resolve/claimdict on certificate_id 2019-01-17 23:24:58 -05:00
Victor Shyba b707ee4844 encode and decode detached signatures from claim dict 2019-01-17 23:24:58 -05:00
Victor Shyba 0e9888be3b fixes from review 2019-01-17 23:24:58 -05:00
Victor Shyba 68b56b7858 adds test for claim_show 2019-01-17 23:24:58 -05:00
Victor Shyba d32697f611 adjust assertions for fee changes 2019-01-17 23:24:58 -05:00
Victor Shyba 3c54b2bcd7 activate detached sigs 2019-01-17 23:24:58 -05:00
Victor Shyba 899b182811 fix getclaimsfortx in server 2019-01-17 23:24:58 -05:00
Victor Shyba 0dfb5e008d add integration test with unserializable claim 2019-01-17 23:24:58 -05:00
Victor Shyba abf94357eb tests and fixes for sig validation on undecode-able claims 2019-01-17 23:24:58 -05:00
Victor Shyba 36aded3830 store original payload for signing and verifying 2019-01-17 23:24:58 -05:00
Victor Shyba d7ebf50602 detached as a parameter instead of class attribute 2019-01-17 23:24:58 -05:00
Victor Shyba 523c4c86a9 change default curve to SECP256k1, pass name parameter 2019-01-17 23:24:58 -05:00
Victor Shyba 74cccdbfc7 simplify detached signing and verifying 2019-01-17 23:24:58 -05:00
Victor Shyba 70471eebfa wallet server and resolve working, functional test 2019-01-17 23:24:58 -05:00
Victor Shyba 6339224980 tests for detached signatures 2019-01-17 23:24:58 -05:00
Victor Shyba fe97f7a119 adds a detached signature attribute to ClaimDict 2019-01-17 23:24:58 -05:00
Victor Shyba f8ed605da2 adds a way to verify detached signatures 2019-01-17 23:24:58 -05:00
Victor Shyba e38ef2af1f adds a way to sign detached 2019-01-17 23:24:58 -05:00
Victor Shyba 5d8e627d02 adds a separate representation for a detached signature 2019-01-17 23:24:58 -05:00
Lex Berezhny c9d4ad5450 pylint unused import 2019-01-17 19:04:52 -05:00
Lex Berezhny 31586bf588 error out when user tries --http-auth 2019-01-17 19:04:52 -05:00
Lex Berezhny 974d3c83e3 keep wallet_balance to test deprecation with, until we find better way to do this 2019-01-17 17:35:15 -05:00
Lex Berezhny 8dc8ca382b removed previously deprecated commands 2019-01-17 17:35:15 -05:00
Lex Berezhny 1dbb250413 fix bytes vs str error in migrate5to6 script 2019-01-15 21:44:16 -05:00
Lex Berezhny 9f9ec692c7 orchstr8 docker for lbry 2019-01-15 17:15:24 -05:00
Thomas Zarebczan 2c1b54c76c Add note about wallet needing to be unlocked (grin was confused about this) 2019-01-15 15:59:45 -05:00
Thomas Zarebczan 7314494fb9 add m4v
It's a container type very similar to mp4. Someone recently added one in the community top bids section.
2019-01-15 15:55:59 -05:00
Jack Robison e74a5e8ef3 cleanup 2019-01-11 16:36:55 -05:00
Jack Robison b9af999739 rename -> time_to_first_byte.py 2019-01-11 16:36:55 -05:00
hackrush 2de155f90f Profile get using rpc calls 2019-01-11 16:36:55 -05:00
hackrush ab3db2e626 Get paths from args 2019-01-11 16:36:55 -05:00
hackrush f9d26b04d8 Get averages! 2019-01-11 16:36:55 -05:00
hackrush 24e0673198 Profile downloading of sd blobs 2019-01-11 16:36:55 -05:00
hackrush 833261801f Profile downloading of sd blobs 2019-01-11 16:36:55 -05:00
hackrush 6c8cbc1b9e added additional fields to uri parser 2019-01-11 15:10:22 -05:00
Lex Berezhny 6ae4e68d1c looping_call shouldnt need to mess with cancellation 2019-01-11 14:33:45 -05:00
Oleg Silkin d22a2dee33 Removes await from shutdown call 2019-01-11 14:33:45 -05:00
Oleg Silkin 0f1ed589a5 Modifies the unit tests for the analytics manager to have async routines
Signed-off-by: Oleg Silkin <o.silkin98@gmail.com>
2019-01-11 14:33:45 -05:00
Oleg Silkin b032e99dd1 Removes the twisted dependency from the analytics file and replaces it
with `asyncio` and `aiohttp`. Also removes the calling loop from the
analytics in favor of a `dict` that stores the names of the methods
and their coroutines that are wrapped inside a looping task. The tasks
are canceled when the analytics manager is asked to shutdown

Signed-off-by: Oleg Silkin <o.silkin98@gmail.com>
2019-01-11 14:33:45 -05:00
Lex Berezhny fd7811f38d v0.30.5rc3 2019-01-11 01:17:43 -05:00
Lex Berezhny 071db3a73d release.py script now extracts backwards-incompatible: value from PR description and adds it to changelog under Backwards Incompatible Changes 2019-01-11 01:16:59 -05:00
Lex Berezhny 950c122850 fix blob announce command argument constraints 2019-01-11 00:54:15 -05:00
Thomas Zarebczan e0386ea9dc rename lbnyum_wallet_dir to wallet_dir 2019-01-11 00:02:52 -05:00
Thomas Zarebczan 6c3eee0f0e Update conf.py 2019-01-10 23:50:59 -05:00
Thomas Zarebczan 4c23add193 fix unavailable download directory
If an uploaded file had a directory that was removed, or if the old download directory doesn't exist, use the default.
2019-01-10 23:05:43 -05:00
Lex Berezhny 248d1a18c7 pylint 2019-01-10 22:38:27 -05:00
Lex Berezhny 17d1b85423 run brew upgrade twice and ignore result 2019-01-10 22:38:27 -05:00
Lex Berezhny b0deb63129 blob_reflect fix 2019-01-10 22:38:27 -05:00
Lex Berezhny ebdb33bd11 blob_availability fix 2019-01-10 22:38:27 -05:00
Lex Berezhny 69ccbda168 fixed error handling when calling commands 2019-01-10 22:38:27 -05:00
Lex Berezhny a38643cca6 remove d2f from storage function call 2019-01-10 22:38:27 -05:00
Lex Berezhny 712b19605e Revert "fix breaking integration tests after fees changed due to content_type modification"
This reverts commit 0613cf2d59.
2019-01-10 10:22:37 -05:00
Alex Grintsvayg 8a5f3842fc fix media type for files without extension 2019-01-10 10:22:37 -05:00
Lex Berezhny db3e578bcd v0.30.5rc2 2019-01-08 23:56:07 -05:00
Lex Berezhny 7e22adff7c added contributor username to each PR listed in changelog 2019-01-08 23:55:34 -05:00
Lex Berezhny 76fbaac658 do not log aiohttp.access 2019-01-08 23:54:32 -05:00
hackrush c4fac29ecb actually get the certificate_id 2019-01-08 23:45:20 -05:00
Lex Berezhny 33392b3351 migration fix 2019-01-08 23:15:25 -05:00
Lex Berezhny 0613cf2d59 fix breaking integration tests after fees changed due to content_type modification 2019-01-08 17:54:40 -05:00
Alex Grintsvayg a930ca0631 if media type is not listed, default to a type that includes the file extension 2019-01-08 16:06:44 -05:00
Lex Berezhny 63a7271008 v0.30.5rc1 2019-01-07 22:15:37 -05:00
Lex Berezhny 5ca0ebcbb2 store loop on storage 2019-01-07 21:56:05 -05:00
Lex Berezhny 2c84b7a7dd get_running_loop() -> get_event_loop() 2019-01-07 21:56:05 -05:00
Lex Berezhny 71a0c704bd storage.clock.seconds() -> asyncio.get_running_loop().time() 2019-01-07 21:56:05 -05:00
Lex Berezhny d30e7cd29e use loop.time() instead of reactor.seconds() in SQLite storage 2019-01-07 21:56:05 -05:00
Lex Berezhny aeb7c6fe76 pylint, removed unused import 2019-01-07 21:56:05 -05:00
Lex Berezhny 134c20da93 stream cost estimate command refactor 2019-01-07 21:56:05 -05:00
Lex Berezhny 893b4fc3ca fixes 2019-01-07 21:56:05 -05:00
Lex Berezhny 9b27e81940 fix logging 2019-01-07 21:56:05 -05:00
Lex Berezhny 72c0acf269 test_HTTPBlobDownloader 2019-01-07 21:56:05 -05:00
Lex Berezhny 7be53ad692 more bug fixes 2019-01-07 21:56:05 -05:00
Lex Berezhny 16e596ec87 lots o bugs fixed discovered while using desktop app 2019-01-07 21:56:05 -05:00
Lex Berezhny 3fdcb80960 minor test fixes 2019-01-07 21:56:05 -05:00
Lex Berezhny 26a3221bac hash announcer fix 2019-01-07 21:56:05 -05:00
Lex Berezhny eb16cb0aa8 fix test_HashBlob 2019-01-07 21:56:05 -05:00
Lex Berezhny b5340505e5 fix blobmanager tests and dont run conf tests 2019-01-07 21:56:05 -05:00
Lex Berezhny 4bb8129bb7 fixing tests to work with asyncio daemon 2019-01-07 21:56:05 -05:00
Lex Berezhny 7f838d3a31 all integration tests run with plain unittest now 2019-01-07 21:56:05 -05:00
Lex Berezhny c186c31bf7 simplified component setup 2019-01-07 21:56:05 -05:00
Lex Berezhny 86fb99a37c fully asyncio daemon api 2019-01-07 21:56:05 -05:00
Lex Berezhny 6c52450858 temporary module providing d2f and f2d for converting between twisted and asyncio 2019-01-07 21:56:05 -05:00
Lex Berezhny fb4e94b04a asyncifying minimal amount of the old blob/p2p code 2019-01-07 21:56:05 -05:00
Lex Berezhny 248baf58b4 asyncio daemon 2019-01-07 21:56:05 -05:00
Lex Berezhny 0845d65f4e rebase master 2019-01-07 21:56:05 -05:00
Lex Berezhny dcc5415648 v0.30.4 2019-01-04 22:03:39 -05:00
Lex Berezhny 3c38a88b22 v0.30.4rc2 2019-01-04 11:53:54 -05:00
Victor Shyba 187b7670b1 adds a dockerfile for wallet server 2019-01-04 11:10:52 -05:00
Victor Shyba 049c17ff08 adds testenet to wallet server 2019-01-04 11:10:52 -05:00
Thomas Zarebczan 8de97bb441 Add lbrynet4 dht seed node (#1734)
added `lbrynet4.lbry.io` to list of default DHT nodes
2019-01-04 11:07:02 -05:00
Lex Berezhny 97c0092923 v0.30.4rc1 2019-01-04 03:14:56 -05:00
Lex Berezhny e65e1fb5a2 added utxo_release command 2019-01-04 03:14:37 -05:00
Victor Shyba 20dd1e55f5 fix data_dir parameter parsing 2019-01-04 01:25:35 -05:00
Lex Berezhny 6aa709e34e v0.30.3 2018-12-21 18:45:34 -05:00
Lex Berezhny e73f121c08 v0.30.3rc1 2018-12-19 09:12:56 -05:00
Victor Shyba 3b9d6c4f8e Instantiate resolver before the request to hold the selected height 2018-12-19 09:04:30 -05:00
Alyssa Callahan 69b60c19c3 Merge pull request #1712 from lbryio/readme-update
Update README.md
2018-12-17 10:47:29 -05:00
Lex Berezhny 04278cb1ab release script automatically figures out prerelease status 2018-12-14 22:25:06 -05:00
Lex Berezhny e8dec5b440 v0.30.2 2018-12-14 21:50:48 -05:00
Lex Berezhny 75f82b107e updated release script with unittests and ability to go from rc to final release 2018-12-14 21:50:30 -05:00
Jack Robison 0e90c4b611 Merge pull request #1717 from lbryio/fix-directory-args
fix directory cli args and update help message
2018-12-14 17:09:15 -05:00
Jack Robison fb66b68bc8 fix directory cli args, update help message 2018-12-14 15:51:11 -05:00
Victor Shyba c601677386 run schema unit tests on travis 2018-12-14 09:21:31 -05:00
Lex Berezhny 1fbd1cd04e v0.30.2rc2 2018-12-12 19:52:11 -05:00
Jack Robison 4a71b91fbf Merge pull request #1671 from lbryio/remove-android-helpers
remove `lbrynet.androidhelpers` module
2018-12-12 18:35:31 -05:00
Jack Robison b8b0d8b1ac refactor default directory setup, remove lbrynet.androidhelpers
fixes https://github.com/lbryio/lbry/issues/1606
2018-12-12 17:46:10 -05:00
Jack Robison 8ef73d3985 update docs 2018-12-12 12:19:26 -05:00
Jack Robison 05e556a5e3 fix generate_json_api script 2018-12-12 12:18:32 -05:00
Lex Berezhny d1da42bea8 Update README.md
* updated description and added bullet points for main features of this SDK
* lex (@eukreign ) is the official product owner of lbrysdk
2018-12-12 11:47:28 -05:00
Lex Berezhny 90685e8357 deploy release for tags 2018-12-12 00:10:50 -05:00
Lex Berezhny a471d5d167 v0.30.2rc1 2018-12-11 23:44:02 -05:00
Lex Berezhny 16122299e4 revert version back 0.30.1 2018-12-11 23:39:08 -05:00
Lex Berezhny 9f18da4bae release script (#1697)
* travis deploying to github
* release script
2018-12-11 23:24:19 -05:00
Lex Berezhny 1abce52867 v0.30.2rc1 2018-12-11 23:09:31 -05:00
Lex Berezhny 072e2b65d3 revert version change 2018-12-11 22:31:45 -05:00
Lex Berezhny f6d11e5467 0.30.1rc1 2018-12-11 22:29:40 -05:00
Oleg Silkin 458bddcb65 Removes Faker from Dependencies (#1687)
* Removes faker import and its usage
* Replaces data generated with faker with static data
* Drops faker as a dependency & removes extras require section
* Removes installation instruction for the test dependencies in setup.py
2018-12-11 20:19:58 -05:00
Victor Shyba 5fad43ff69 fix test tear down and unify test_commands 2018-12-11 16:34:37 -05:00
Victor Shyba 6992d0d947 add new test to tox.ini 2018-12-11 16:34:37 -05:00
Victor Shyba d1b0180437 add tests for claim_list_by_channel 2018-12-11 16:34:37 -05:00
Victor Shyba ba59e126cd forward errors outside of uris on channel listing 2018-12-11 16:34:37 -05:00
Victor Shyba 4e3467bb7e fix windows release type set script 2018-12-11 16:30:35 -05:00
Lex Berezhny 4d4c705eb9 0.30.1 changelog updates 2018-12-09 23:30:09 -05:00
Lex Berezhny ce7b3184c4 0.30.1 2018-12-09 23:15:11 -05:00
Lex Berezhny b71faddaac upload zip files into the same directory 2018-12-08 19:01:13 -05:00
Lex Berezhny 1328cb4cce check file exists before reading config 2018-12-08 19:01:13 -05:00
Lex Berezhny 946488bce9 correctly save daemon_settings.yml on fresh install 2018-12-08 19:01:13 -05:00
Victor Shyba 7713881e71 additional tests for resolution border cases 2018-12-08 14:40:37 -05:00
Victor Shyba 26fcc48c5a fix for name collision on channel path 2018-12-08 14:40:37 -05:00
Thomas Zarebczan bf12fb68a2 use xcode8.3 for mac build (#1692)
use xcode8.3 for mac build
2018-12-08 14:38:35 -05:00
Jack Robison 3a4c9ff4f9 bump to 0.30.1rc11 2018-12-07 13:07:06 -05:00
Jack Robison 6180a2917d add epub and wmv content types 2018-12-07 13:06:36 -05:00
Lex Berezhny 014cbb4623 v0.30.1rc10 2018-12-07 12:04:28 -05:00
Jack Robison ee98251d79 Merge pull request #1700 from lbryio/mimetypes
improve contentType detection
2018-12-07 12:02:41 -05:00
Jack Robison e3d08ec773 improve contentType detection
fixes https://github.com/lbryio/lbry/issues/1690
2018-12-07 11:31:05 -05:00
Lex Berezhny f03e68e63e 0.30.1rc9 2018-12-06 00:51:07 -05:00
Lex Berezhny 72090b45dd confirmations is returned as int 2018-12-06 00:49:07 -05:00
Lex Berezhny 93f9622798 fix json_response_encoder to also return correct confirmations 2018-12-06 00:49:07 -05:00
Lex Berezhny 16ff4b4e2c confirmations fix 2018-12-06 00:49:07 -05:00
Lex Berezhny c90cca4b20 fix #1683 2018-12-05 14:38:35 -05:00
Lex Berezhny 98d83ea74f fix for integration test 2018-12-05 12:14:43 -05:00
Lex Berezhny a234f92d2c 0.30.1rc8 2018-12-05 11:53:37 -05:00
Lex Berezhny 8ba3575901 dropped aiorpcx 2018-12-05 09:17:36 -05:00
Jack Robison a857c83c59 Merge pull request #1685 from lbryio/fix-1674
fix int too large to convert to integer when updating claim
2018-12-04 18:16:24 -05:00
Jack Robison a6128ac8a2 fix https://github.com/lbryio/lbry/compare/fix-1674 2018-12-04 17:35:40 -05:00
hackrush 2693301ad0 Fix python and pip3 not available in xcode8 2018-12-04 15:12:07 -05:00
Victor Shyba f7c7cb9535 fix transaction_show 2018-12-04 12:12:08 -05:00
Jack Robison b36a5f9b6b Merge pull request #1678 from lbryio/warn_migration_bad_claim_id
check and log claim id validity during migration
2018-11-30 15:58:26 -05:00
Jack Robison 4826eb9969 migrate double hex encoded certificates 2018-11-30 14:00:35 -05:00
Victor Shyba c34b150794 check and log claim id validity during migration 2018-11-30 14:00:35 -05:00
Lex Berezhny c4c7670a7c do not run brew update 2018-11-30 12:20:07 -05:00
Lex Berezhny 0918bac86e mac build stuff 2018-11-30 11:50:32 -05:00
Victor Shyba a60a921706 resolve claims on claim_list (#1483) 2018-11-30 11:46:56 -05:00
Lex Berezhny b3ee29b66d maybe python is already installed on mac 2018-11-30 11:30:03 -05:00
Lex Berezhny 3a1a9b988c on mac brew, python 3 is installed with plain python while python2 now requires python@2 2018-11-30 10:55:15 -05:00
Lex Berezhny 12f2bebc46 build for mac using xcode8 2018-11-30 10:34:45 -05:00
Jack Robison 31a3812c8a bump 2018-11-29 17:23:32 -05:00
Lex Berezhny 70ec08e30b icons! 2018-11-29 17:15:18 -05:00
Jack Robison 23e8f0d849 Merge pull request #1676 from lbryio/fix-1573
unlock wallet after encrypting it
2018-11-29 16:13:21 -05:00
Jack Robison 1cdfcd5b4e unlock wallet after encrypting it 2018-11-29 15:16:53 -05:00
Lex Berezhny dc2786fbc0 assert instead of if 2018-11-28 18:03:33 -05:00
Lex Berezhny 7e39b0c4fe run ClaimManagement.test_create_update_and_abandon_claim 2018-11-28 18:03:33 -05:00
Lex Berezhny 4af830c405 tests for transaction_list going through claim creation -> claim update -> claim abandon 2018-11-28 18:03:33 -05:00
Lex Berezhny fe844581fd correctly split value/fee in transaction_list 2018-11-28 18:03:33 -05:00
Lex Berezhny 443ce6352b typo 2018-11-28 18:03:33 -05:00
Lex Berezhny 158c354a27 split out fee from value when possible 2018-11-28 18:03:33 -05:00
Lex Berezhny 92d1230a90 remove fees from value, abandon is positive 2018-11-28 18:03:33 -05:00
Jack Robison d64916a061 bump 2018-11-27 16:28:55 -05:00
Alyssa Callahan 20a0568253 Merge pull request #1567 from preethamvishy/master
Update pull request template
2018-11-27 15:45:54 -05:00
Alyssa Callahan 5a1f42ee54 Merge branch 'master' into master 2018-11-27 15:39:48 -05:00
Alyssa Callahan ab95f14e21 Update PULL_REQUEST_TEMPLATE.md 2018-11-27 15:38:06 -05:00
Jack Robison bae2676b69 Merge pull request #1669 from lbryio/fix_invalid_sigs_from_name
allow invalid sigs claims to show outside a channel
2018-11-27 14:41:28 -05:00
Victor Shyba ea6869a63e allow invalid sigs claims to show outside a channel 2018-11-27 15:36:53 -03:00
Jack Robison 953327c150 bump 2018-11-27 09:52:33 -05:00
Lex Berezhny e01ad6bc85 disable travis cache on mac build 2018-11-26 20:55:03 -05:00
Victor Shyba 47a33009ff remove python 3.6 steps from travis 2018-11-26 20:55:03 -05:00
Victor Shyba bc844013a1 (win build) add pywin32 and netifaces wheels 2018-11-26 20:55:03 -05:00
Victor Shyba 2fd2319cdb use lbry dockerhub image 2018-11-26 20:55:03 -05:00
Victor Shyba a50657c472 add dockerfile and readme on how win docker builder is generated 2018-11-26 20:55:03 -05:00
Victor Shyba ef5f6e06ba use vshyba/temporary_pyinstaller_32bits_py37:latest 2018-11-26 20:55:03 -05:00
Victor Shyba 977cf9f506 log python version on windows build script 2018-11-26 20:55:03 -05:00
Lex Berezhny ffae10d11f drop pytest 2018-11-26 18:02:12 -05:00
Thomas Zarebczan 221107e76e test mac build, do not merge (#1659)
* Update .travis.yml to use xcode 6.4 for mac build
2018-11-26 14:13:45 -05:00
Jack Robison e74c78b607 Merge pull request #1432 from lbryio/get-block-on-bytes-written
[API] block get on bytes being written to the output file
2018-11-26 12:16:42 -05:00
Jack Robison 434c7b57b1 tests 2018-11-26 11:36:28 -05:00
Jack Robison 9dc4608a62 combine StreamProgressManager and FullStreamProgressManager
-add wrote_first_data Deferred to FullStreamProgressManager
2018-11-26 11:36:28 -05:00
Jack Robison fc69366092 set the download manager before creating the first deferred 2018-11-26 11:36:28 -05:00
Victor Shyba 9f0b1f3e25 fix pylint failures unveiled after astroid pin 2018-11-26 11:32:41 -05:00
Victor Shyba 3f03a845ec only return invalid signatures if its a direct resolution 2018-11-26 11:32:41 -05:00
Victor Shyba bc28976e12 adds test for claims on abandoned channels 2018-11-26 11:32:41 -05:00
Victor Shyba 8f826098f9 validate claim id before resolving 2018-11-26 11:32:41 -05:00
Victor Shyba 0a765e6207 handle errors on resolve 2018-11-26 11:32:41 -05:00
Lex Berezhny f83aa61923 force astroid==2.0.4 in .travis.yml 2018-11-26 10:47:48 -05:00
Oleg Silkin f9ff3e5d14 Removes third-party mock dependency (#1661)
* Removes mock from setup.py dependencies
* Refactors name resetTime to reset_time
* Replaces import mock with built-in unittest.mock
* Refactors imports to separate built-in, external, and project modules
2018-11-25 11:12:30 -05:00
Oleg Silkin b3fde9d78d Removes six From Project (#1660)
* Replaces `six` urllib with python 3's urllib
* Replaces `six`'s int2byte method with native `bytes` class
* Removes edge case testing for python2 vs python3
* Removes useless object inheritance
* Uses native io.BytesIO instead of six.BytesIO
* Removes six from dependencies
2018-11-22 23:24:21 -05:00
Lex Berezhny 0c999083cc added jsonrpc_account_send and dropped torba[server] 2018-11-21 01:14:39 -05:00
Victor Shyba 561215db8a fix download script 2018-11-21 00:25:43 -05:00
Jack Robison 8dbb7f2c78 bump 2018-11-20 21:37:49 -05:00
Lex Berezhny 6f7f3bb316 fixes #1653 2018-11-20 16:12:51 -05:00
Jack Robison 8d507c4b73 bump 2018-11-19 21:12:39 -05:00
Lex Berezhny 2beb0eda94 SupportingSupports test does not exist 2018-11-19 20:34:38 -05:00
Lex Berezhny b7e07b72ca update tox.ini to run correct tests 2018-11-19 20:34:38 -05:00
Lex Berezhny fde8718c8c make sure only actual claims/updates are found when updating a claim or sending it to someone 2018-11-19 20:34:38 -05:00
Lex Berezhny b2d1bddcfe update to latest torba 2018-11-19 20:34:38 -05:00
Lex Berezhny dd1d1be994 working claim_send_to_address 2018-11-19 20:34:38 -05:00
Jack Robison a93957178a bump 2018-11-19 17:31:31 -05:00
Jack Robison 5026d0b2f3 Merge branch 'fix-get-bad-claim-error' 2018-11-19 17:31:01 -05:00
Jack Robison cc8b47d882 improve error when get fails to decode the resolved claim 2018-11-19 17:30:02 -05:00
Jack Robison db0b5961b3 Merge pull request #1642 from jessopb/readmeStuff
adds daemon config to and updates links in readme
2018-11-19 14:37:15 -05:00
Jessop Breth cb6f7ada42 adds sample config and links to settings documentation 2018-11-19 10:10:02 -05:00
Jack Robison 59464099b4 add example daemon_settings.yml 2018-11-19 09:42:13 -05:00
preserveddarnell aed00b53fd Update various typos & edits 2018-11-18 20:52:18 -05:00
YULIUS KURNIAWAN KRISTIANTO cdb0f97cb4 added more social links 2018-11-18 10:29:47 -05:00
Jack Robison 33bcc8256c Merge branch 'fix-1624' 2018-11-16 11:45:05 -05:00
Jack Robison 0a8d44f12f Merge branch 'populate-buckets' 2018-11-16 11:44:25 -05:00
Jack Robison 1b74594f5b don't create ssl/auth keys if disabled
fixes https://github.com/lbryio/lbry/issues/1624
2018-11-16 11:12:33 -05:00
Jack Robison b0718d295e fix empty buckets in routing_table_get 2018-11-16 10:59:12 -05:00
Jack Robison 9bb91b126d use midpoint id instead of random for refresh list 2018-11-16 10:59:12 -05:00
Jack Robison 6dbb00d8d5 populate buckets when joining the dht 2018-11-16 10:59:12 -05:00
Jack Robison 3a27a4509a Merge pull request #1639 from lbryio/fix-1638
fix bytes/str in iterativeFindValue
2018-11-14 16:22:30 -05:00
Jack Robison 1d438f5d23 fixes https://github.com/lbryio/lbry/issues/1638 2018-11-14 15:35:59 -05:00
YULIUS KURNIAWAN KRISTIANTO 3840f19869 fix mistake code 2018-11-14 14:20:02 -05:00
Lex Berezhny ec0483b975 header progress percent starts at 0 instead of None 2018-11-12 21:16:05 -05:00
Jack Robison 2fba1f417f less verbose logging 2018-11-12 14:47:11 -05:00
Jack Robison 5b419b55c5 fix peer_list 2018-11-12 14:45:41 -05:00
Victor Shyba b31379e51c log and handle misformed certificate claims from #1609 2018-11-10 20:49:36 -05:00
Victor Shyba 8394dcb663 log and handle misformed claims from #1611 2018-11-10 20:49:36 -05:00
Lex Berezhny 457a56ce04 Merge pull request #1604 from lbryio/fix-external-ip
Fix setting the external ip
2018-11-10 20:48:40 -05:00
Jack Robison 7c8016d807 remove accidentally added lbrynet.schema.signature 2018-11-09 16:52:46 -05:00
Jack Robison d704336f20 absolute imports 2018-11-09 16:52:46 -05:00
Jack Robison 1b4230b06e move is_valid_blobhash to lbrynet.blob.blob_file 2018-11-09 16:52:46 -05:00
Jack Robison 88095af1cd move system_info.py into lbrynet.extras 2018-11-09 16:52:46 -05:00
Jack Robison e9efd9f68a improve get_external_ip 2018-11-09 16:52:46 -05:00
Jack Robison 7b32e2ff7c don't block the dht component on populating the routing table 2018-11-09 16:52:46 -05:00
Jack Robison 572777305e reduce analytics event frequency 2018-11-09 16:52:46 -05:00
Jack Robison 7e27a6e37d fix 2018-11-09 16:52:46 -05:00
Jack Robison b98621d7ea avoid name errors 2018-11-09 16:52:46 -05:00
Jack Robison bc5824b8bc better setup of custom logger class 2018-11-09 16:52:46 -05:00
Jack Robison a6b643bb34 move files 2018-11-09 16:52:42 -05:00
Jack Robison 488b976734 move HashAnnouncer.py to extras 2018-11-09 14:16:33 -05:00
Jack Robison 253912b52e fix skipping dht, peer_protocol_server, and hash_announcer components
-move PeerFinder and PeerManager initialization to the ComponentManager

-remove dht component requirement from all but the hash_announcer component. This allows running the file manager component without either of the server components.
2018-11-09 14:16:33 -05:00
Jack Robison 0bf8416d9f fix components_to_skip setting 2018-11-09 14:16:33 -05:00
Jack Robison 2e921437d8 fix external ip setup
fixes https://github.com/lbryio/lbry/issues/1561
2018-11-09 14:16:33 -05:00
Jack Robison b6f9b60683 remove commit in version for source installs 2018-11-09 14:16:33 -05:00
Lex Berezhny 57430e9f5f fixed #1616 2018-11-09 12:03:55 -05:00
Victor Shyba 1e13fbee8d await missing on resolve.py 2018-11-09 11:11:57 -05:00
Lex Berezhny 3d8b9e2b66 block=False for abandon during erics epic adventure tests 2018-11-07 18:28:02 -05:00
Lex Berezhny 64898d43f1 tearDown() should run before daemon shutdown 2018-11-07 18:28:02 -05:00
Lex Berezhny b2117fddab on_transaction -> ledger.wait 2018-11-07 18:28:02 -05:00
Lex Berezhny 5658623f29 claims include supports 2018-11-07 18:28:02 -05:00
Lex Berezhny 934682bd0f --blocking support in abandon command 2018-11-07 18:28:02 -05:00
Victor Shyba daecc582f3 simplify bind statement 2018-11-07 11:29:56 -05:00
Victor Shyba 3cb4012dbf unify loop 2018-11-07 11:29:56 -05:00
Victor Shyba f7bd44a4f5 fix query for windows (999 vars limit) 2018-11-07 11:29:56 -05:00
hackrush 52317b05bb Better help message instructions 2018-11-07 11:02:20 -05:00
Lex Berezhny 83c0baa3cb test abandoning claim at a loss, required a fix in torba 2018-11-07 10:57:11 -05:00
Lex Berezhny 340c5addab insufficientfundserror test 2018-11-07 10:57:11 -05:00
Lex Berezhny e3fd94dbf0 pylint fix 2018-11-07 10:57:11 -05:00
Lex Berezhny 923fcfb9e9 run publish integration tests separately 2018-11-07 10:57:11 -05:00
Lex Berezhny 0ae5f1a4c6 added fix for #1533 2018-11-07 10:57:11 -05:00
Lex Berezhny 90e06abba2 fixes for #1569 2018-11-07 10:57:11 -05:00
Victor Shyba 6ecfc94ff1 add balance to startup report 2018-11-07 10:50:13 -05:00
Alyssa Callahan 60c4a727d3 Merge pull request #1574 from czakarian/patch-1
Fix typos / grammar changes
2018-11-07 10:19:59 -05:00
Alyssa Callahan 9fbb9f2e98 Merge branch 'master' into patch-1 2018-11-07 10:18:58 -05:00
Lex Berezhny f1feec66c0 fixed path to cli.py in setup.py 2018-11-06 13:55:22 -05:00
Lex Berezhny 73b9b983b1 pinned protobuf to specific version, fixes #1602 2018-11-06 00:05:40 -05:00
Victor Shyba 588779e3a4 makefile for using types repo 2018-11-05 20:48:21 -05:00
Jack Robison b7ae446059 Merge pull request #1495 from lbryio/retry_dht
[DHT] retry on protocol level
2018-11-05 15:16:53 -05:00
Victor Shyba db898307b5 obey the rpcAttempts constant on retries 2018-11-05 15:16:39 -03:00
Victor Shyba 7e3512cbf2 retry on protocol level 2018-11-05 15:16:39 -03:00
Lex Berezhny cdd65c6783 updated pyinstaller build instructions 2018-11-04 20:25:24 -05:00
Lex Berezhny 320ad717cd pylint cleanup 2018-11-04 20:25:24 -05:00
Lex Berezhny 264e6744e1 updated references after moving conf to lbrynet.extras.daemon.conf 2018-11-04 20:25:24 -05:00
Lex Berezhny 64e2b79c07 moved conf into daemon 2018-11-04 20:25:24 -05:00
Lex Berezhny 9255bb5abe updated references after moving analytics to lbrynet.extras.daemon.analytics 2018-11-04 20:25:24 -05:00
Lex Berezhny d7525dacc3 moved analytics to lbrynet/extra/daemon 2018-11-04 20:25:24 -05:00
Lex Berezhny 5cb07100f8 deleted unused interfaces file 2018-11-04 20:25:24 -05:00
Lex Berezhny 171c080d33 updated references lbrynet.file_manager -> lbrynet.blob 2018-11-04 20:25:24 -05:00
Lex Berezhny cfc3ec2a05 merged lbrynet/file_manager into lbrynet/blob 2018-11-04 20:25:24 -05:00
Lex Berezhny 1392b8f83b removed lbrynet/lbry_file/__init__.py 2018-11-04 20:25:24 -05:00
Lex Berezhny 8b265801df updated references lbrynet.lbry_file.client -> lbrynet.file_manager.client 2018-11-04 20:25:24 -05:00
Lex Berezhny 6c4fb8dab1 moved lbrynet/lbry_file/client into lbrynet/file_manager/client 2018-11-04 20:25:24 -05:00
Lex Berezhny c1a9cd93ee updated references lbrynet.crypstream -> lbrynet.blob 2018-11-04 20:25:24 -05:00
Lex Berezhny 0d4e83e167 merged lbrynet/cryptstream into lbrynet/blob 2018-11-04 20:25:24 -05:00
Lex Berezhny 1292eb345d fix dbmigrator pylint issue 2018-11-04 20:25:24 -05:00
Lex Berezhny f4c6dc90c9 updated references lbrynet.core -> lbrynet.p2p 2018-11-04 20:25:24 -05:00
Lex Berezhny 8c8f5d8f58 renamed lbrynet/core -> lbrynet/p2p 2018-11-04 20:25:24 -05:00
Lex Berezhny b6d986cb40 updated references after moving lbrynet/database into lbrynet/extras/daemon 2018-11-04 20:25:24 -05:00
Lex Berezhny 6cc134c7c8 merged lbrynet/database with lbrynet/extras/daemon 2018-11-04 20:25:24 -05:00
Lex Berezhny eee964cfa0 updated references to daemon after moving it to extras 2018-11-04 20:25:24 -05:00
Lex Berezhny a03c94bfad moved daemon to extras/daemon 2018-11-04 20:25:24 -05:00
Lex Berezhny f1f0917063 updated all references to point to new reflector location 2018-11-04 20:25:24 -05:00
Lex Berezhny fe2476b5aa moved reflector into extras 2018-11-04 20:25:24 -05:00
Lex Berezhny 9c5fa8bf76 winhelpers/winpaths.py -> extras/winpaths.py 2018-11-04 20:25:24 -05:00
Lex Berezhny cd43a71172 update winpaths.py references 2018-11-04 20:25:24 -05:00
Lex Berezhny e2c6de76f4 renamed knownpaths.py -> winpaths.py 2018-11-04 20:25:24 -05:00
Lex Berezhny 05aeadc9fe extraneous LICENSE file since the whole project is already MIT, moved copyright directly into file header 2018-11-04 20:25:24 -05:00
Lex Berezhny 7153417372 #egg=torba[server] 2018-11-04 10:50:15 -05:00
Lex Berezhny fceb2ae0a2 hmm 2018-11-04 10:50:15 -05:00
Lex Berezhny 0cef2fdd82 install extras from github 2018-11-04 10:50:15 -05:00
Lex Berezhny 9c91b135bd pylint requires [wallet-server] 2018-11-04 10:50:15 -05:00
Lex Berezhny e3b127192c install .[test] includes the [wallet-server] 2018-11-04 10:50:15 -05:00
Lex Berezhny ee1738da6e lbrynet.extras.wallet.server pylint fixes 2018-11-04 10:50:15 -05:00
Lex Berezhny 8ca2b75c80 lbryumx -> lbrynet.extras.wallet.server 2018-11-04 10:50:15 -05:00
Lex Berezhny dc81b6def1 install lbryumx after lbrynet so that we dont get the old protobuf 2018-11-04 10:50:15 -05:00
Lex Berezhny 22f5aaee70 regenerated protobufs 2018-11-04 10:50:15 -05:00
Lex Berezhny 4bd6569c9c lbrynet.schema.schema 2018-11-04 10:50:15 -05:00
Lex Berezhny dbf39baeb1 fix lbrynet.schema import 2018-11-04 10:50:15 -05:00
Lex Berezhny 3adf3c26f2 add set_build.py back in 2018-11-04 10:50:15 -05:00
Lex Berezhny fa0d13e630 protobuf==3.2.0 2018-11-04 10:50:15 -05:00
Lex Berezhny 227a82d6d3 update protobuf version 2018-11-04 10:50:15 -05:00
Lex Berezhny 65cfc1f63c removed lbryschema references from scripts and copied lbryschema requirements into local setup.py 2018-11-04 10:50:15 -05:00
Victor Shyba e46f0e2444 fix namespace lbryschema->lbrynet.schema 2018-11-04 10:50:15 -05:00
Victor Shyba 00bc7995c4 move lbryschema/tests/ into tests/unit/schema/ 2018-11-04 10:50:15 -05:00
Victor Shyba 2dcebe48b7 move lbryschema/lbryschema into lbrynet/schema 2018-11-04 10:50:15 -05:00
Lex Berezhny d12b6cddb3 fix integration tests 2018-11-04 02:28:36 -05:00
Lex Berezhny 699c057f2d upgrading to new torba 2018-11-04 02:28:36 -05:00
Lex Berezhny 3ce9f31f4b recursive channel lookup 2018-11-04 02:28:36 -05:00
Lex Berezhny 485d5d55e8 pylint fix 2018-11-04 02:28:36 -05:00
Lex Berezhny 6ee9f2513b add extra debug logging when a channel is not a claim errors 2018-11-04 02:28:36 -05:00
Daniel Brereton 81a6e14a5c added alt text 2018-11-02 19:38:30 -04:00
czakarian 496ffe0096 Fix typos / grammar changes 2018-10-31 16:35:39 -07:00
Preetham Viswanathan 216c760179 Merge pull request #1 from preethamvishy/preethamvishy/pr-template
Update pull request template
2018-10-30 17:16:56 -04:00
Preetham Viswanathan 01eca3004f Update PULL_REQUEST_TEMPLATE.md 2018-10-30 16:57:41 -04:00
GwanYeong Kim 2f47a3758e Remove unused Python modules in tests/unit/test_cli.py 2018-10-30 10:32:34 -04:00
pushkyn 73bcc1796f Add GitHub pull request template 2018-10-30 10:27:47 -04:00
Jack Robison e03e41ad31 bump 2018-10-26 16:51:23 -04:00
Jack Robison 6cb30f8e05 Merge branch 'fix-update-stream'
[API] add `aioupnp_version` to `upnp` field in the response to `status`
[API] fix uncaught claim signature errors
2018-10-26 16:49:51 -04:00
Jack Robison d9b1be6214 add aioupnp version to upnp status
-set non-blank not found message if upnp fails
2018-10-26 16:48:52 -04:00
Jack Robison 5f2f06c269 fix loggly 2018-10-26 13:01:50 -04:00
Jack Robison e1dcdcf4db catch claim signature error 2018-10-26 13:01:50 -04:00
Jack Robison 416472c5d8 log stream update failure 2018-10-26 13:01:46 -04:00
Alyssa Callahan 196f3ba92d added Internal use section 2018-10-24 15:47:07 -04:00
Lex Berezhny 6feae93c58 Update lbrynet/wallet/claim_proofs.py 2018-10-23 12:02:10 -04:00
Lex Berezhny 203d30d4f7 Update lbrynet/wallet/claim_proofs.py 2018-10-23 12:02:10 -04:00
Hugo d0a7ca841b Upgrade unit tests to use more useful asserts 2018-10-23 12:02:10 -04:00
Hugo 2e6c848c39 Upgrade Python syntax with pyupgrade --py36-plus 2018-10-23 12:02:10 -04:00
Jack Robison 18a0204105 set_build regex 2018-10-22 22:27:57 -04:00
Jack Robison 01baa193d5 fix set_build 2018-10-22 20:58:52 -04:00
Jack Robison ba57af5502 fix set_build 2018-10-22 17:55:32 -04:00
Jack Robison 8139f93ff1 set build type 2018-10-22 16:31:05 -04:00
Jack Robison 4596d5c26f bump 2018-10-22 09:12:49 -04:00
Nikita Titov 810a63782d Fixed long description of package 2018-10-20 10:15:29 -04:00
Nikita Titov ea83827d57 Update macOS naming 2018-10-20 10:02:24 -04:00
Jack Robison e19f8f3d95 bump 2018-10-19 12:28:08 -04:00
Jack Robison 7af501e0bc Merge branch 'temp-blob-pass-sqlite-object' 2018-10-19 12:27:27 -04:00
hackrush a692b6ff7d Pass SQLite storage object to DiskBlobManager for single(temp) blob download 2018-10-19 10:39:39 -04:00
Jack Robison 3364ac0976 bump 2018-10-18 18:22:46 -04:00
Jack Robison cf1157f02a Merge branch 'upnp-status' 2018-10-18 18:21:53 -04:00
Jack Robison 0ed56bbe1a tests 2018-10-18 18:12:50 -04:00
Jack Robison 213bdac55f check upnp redirects and external ip in a looping call
-add upnp setup success/fail analytics event
2018-10-18 17:41:49 -04:00
Jack Robison 6d8ab97ca9 fix analytics 2018-10-18 17:41:17 -04:00
Jack Robison fb34476cd7 add upnp status dictionary to status response
-log upnp debugging information as an error
2018-10-18 15:10:00 -04:00
Jack Robison 07d353a947 bump 2018-10-18 13:19:25 -04:00
Jack Robison 3c5fb6490d catch errors 2018-10-18 13:00:27 -04:00
Jack Robison a5d21081b9 Merge pull request #1549 from hugovk/link-to-key
Add link to GPG key
2018-10-18 12:41:42 -04:00
Jack Robison ed98a3cc21 Merge branch 'master' into link-to-key 2018-10-18 12:39:39 -04:00
Jack Robison 45c2cd2986 Merge pull request #1550 from hugovk/fix-spelling
Fix spelling
2018-10-18 12:37:56 -04:00
Hugo 75f21950da Add link to GPG key
And fix some typos.
2018-10-18 14:44:48 +03:00
Hugo 3f704be85c Fix spelling 2018-10-18 14:40:37 +03:00
Jack Robison 67a15ea788 bump 2018-10-18 00:34:58 -04:00
Jack Robison 55bcd2dfee Merge branch 'fix-asyncio-build' 2018-10-18 00:34:04 -04:00
Jack Robison d9757fdae8 remove asyncio from requirements 2018-10-18 00:32:44 -04:00
Jack Robison a4ae8d7c53 bump 2018-10-17 21:36:54 -04:00
Jack Robison 78078b4c9f Merge remote-tracking branch 'origin/fix_dht_logging' 2018-10-17 21:26:46 -04:00
Victor Shyba 1331e2bbe6 fix DHT logging 2018-10-17 22:25:41 -03:00
Lex Berezhny 4c4d652f1b fixes for #1526 and #1483 2018-10-17 21:10:23 -04:00
Lex Berezhny 73bd76b631 format publish output bid value in lbc instead of dewies 2018-10-17 19:50:31 -04:00
Lex Berezhny 59a0f9f602 removed the max response size from Known Issues as it was fixed 2018-10-17 19:38:14 -04:00
Lex Berezhny 1b1847bc31 added Known Issues section 2018-10-17 19:38:14 -04:00
Lex Berezhny 21a569edcc updated CHANGELOG for v0.30 2018-10-17 19:38:14 -04:00
Lex Berezhny 2ddf1a08f6 publish command supports custom account list to lookup channels 2018-10-17 19:37:54 -04:00
Victor Shyba 8754c6fabe fix resolve_name for asyncio 2018-10-17 17:17:25 -04:00
Victor Shyba f049109a38 fix claim_show 2018-10-17 17:17:25 -04:00
Victor Shyba 8db7c15aa1 fix address query handler 2018-10-17 15:33:22 -04:00
Victor Shyba 90e0694d2e fix download from peer script 2018-10-17 15:33:22 -04:00
Victor Shyba 4748cbf654 fix tests for cost estimate 2018-10-17 14:16:00 -04:00
Victor Shyba 74eb991983 fix paid downloads 2018-10-17 14:16:00 -04:00
Victor Shyba e11bae2dda stream cost estimate fix for uri without size 2018-10-17 14:16:00 -04:00
Victor Shyba 8462f56b13 fix blob_list for uris 2018-10-17 14:16:00 -04:00
Victor Shyba 0bec3c7294 fix claim_list_by_channel 2018-10-17 14:16:00 -04:00
Victor Shyba 2bb4052193 fix claim_list 2018-10-17 14:16:00 -04:00
Lex Berezhny 101a3f2b22 respect --show_seed argument to account_list command 2018-10-17 13:45:01 -04:00
Lex Berezhny 04a2bde8dd removed extraneous asyncio import 2018-10-17 13:29:28 -04:00
Jack Robison a1368b05da txupnp --> aioupnp
-update INSTALL.md
2018-10-17 13:29:28 -04:00
Victor Shyba 99ea7a26b0 fix get for asyncio torba 2018-10-17 10:27:57 -04:00
Lex Berezhny 8b3a457925 improve integration test reliability 2018-10-16 21:28:47 -04:00
Lex Berezhny e02e2a249a fix wallet component 2018-10-16 17:09:21 -04:00
Lex Berezhny 3019a1c222 revert tox config 2018-10-16 17:09:21 -04:00
Lex Berezhny 1f3a19c100 pylint 2018-10-16 17:09:21 -04:00
Lex Berezhny 5eac4247e1 working integration tests 2018-10-16 17:09:21 -04:00
Lex Berezhny 1ca82b9d52 major facepalm 2018-10-16 17:09:21 -04:00
Lex Berezhny 21702ec43c asyncio progress for victor 2018-10-16 17:09:21 -04:00
Lex Berezhny d2083bc17e for victor 2018-10-16 17:09:21 -04:00
Lex Berezhny 52edf880e0 fix typo in .travis.yml file 2018-10-16 17:09:21 -04:00
Lex Berezhny 1f45af991f install dependencies needed for asyncio unit tests 2018-10-16 17:09:21 -04:00
Lex Berezhny 8afdd0b2e9 pylint fix 2018-10-16 17:09:21 -04:00
Lex Berezhny 99efc70ba3 run wallet unit tests separate from the other twisted trial based tests 2018-10-16 17:09:21 -04:00
Lex Berezhny 64e306801d updated wallet to use asyncio 2018-10-16 17:09:21 -04:00
Victor Shyba 330db61b36 report balance and summary on startup 2018-10-12 20:53:31 -04:00
Lex Berezhny 357763ff61 respect blockchain_name in daemon settings when creating initial wallet 2018-10-12 12:28:39 -04:00
Lex Berezhny 8a6e6b4d77 add category attribute in JSON RPC output for backwards compatibility 2018-10-12 11:08:01 -04:00
Lex Berezhny 64913c4288 JSON RPC output for TXOs now includes: height, confirmations and valid_signature 2018-10-12 10:41:49 -04:00
Victor Shyba f01ad0e4bd use smart_decode on transaction parsing to support legacy claims 2018-10-11 17:38:23 -04:00
Victor Shyba d9fcc458dc refactor part of peer logic out of try/catch 2018-10-10 15:27:03 -04:00
Victor Shyba 24a11d3f21 make peer search asynchronous 2018-10-10 15:27:03 -04:00
Victor Shyba 234eaf8890 fix download_blob_from_peer script 2018-10-10 15:27:03 -04:00
Tomasz Kopeć 9e2e53147e --conf option improvements (#1455)
* Conf file improvements
* Add test for loading config file
2018-10-10 13:44:51 -04:00
Victor Shyba fc41af5889 set wallet max_gap after sync 2018-10-10 13:38:14 -04:00
Victor Shyba 72ce0cd0b0 explicitly define a default for signatures as expected on 0.21 2018-10-10 11:36:20 -04:00
Jack Robison e4497c1327 Merge remote-tracking branch 'origin/fix-1414' 2018-10-10 11:16:45 -04:00
hackrush a1bf832ea6 fix blob_availability 2018-10-10 16:21:54 +05:30
Lex Berezhny d4e76500fb removed unused import 2018-10-10 00:02:50 -04:00
Lex Berezhny f898bb3d0e database index fix 2018-10-09 23:58:32 -04:00
Lex Berezhny ad13282494 txo (claim_id) column index 2018-10-09 23:40:02 -04:00
Lex Berezhny 73519f8c6a claim_name updated to use new account.get_claims() method 2018-10-09 21:46:54 -04:00
Lex Berezhny b2c95f35c7 fixed test_transaction reliability 2018-10-09 21:32:32 -04:00
Lex Berezhny 0bfccb86ff refactored pagination per @seanyesmunt feedback 2018-10-09 20:46:41 -04:00
Lex Berezhny 5118240897 fix log message in header downloader to log local_height as integer instead of float 2018-10-09 17:39:28 -04:00
Lex Berezhny 427fc179ac fix for #1523 2018-10-09 16:48:12 -04:00
Lex Berezhny 55bcc520e4 pagination method fixes and not showing fee for recieved TX in get_history 2018-10-09 14:46:44 -04:00
Lex Berezhny bc8251696e fix cli version test 2018-10-09 11:37:49 -04:00
Lex Berezhny 677ba25ccd use .format() instead of string concatenation 2018-10-09 11:15:11 -04:00
Alex Grintsvayg 95436b233c fix lbrynet --version 2018-10-09 11:15:11 -04:00
Lex Berezhny 82871b8f6b fix command line argument for account_balance and other paginated commands 2018-10-09 10:30:30 -04:00
Lex Berezhny 9525b87224 inlineCallbacks fix 2018-10-08 22:44:00 -04:00
Lex Berezhny f06098978c ledger.get_transaction dropped in favor of ledger.db.get_transaction 2018-10-08 19:51:44 -04:00
Jack Robison 141491f2da bump version 2018-10-08 16:11:36 -04:00
Victor Shyba 38006f7d29 migrate addresses before starting the migrated wallet 2018-10-08 15:29:57 -04:00
Lex Berezhny 3664c25d98 changed lbry account.get_balance() to have same confirmations default as baseaccount.get_balance() 2018-10-08 15:02:52 -04:00
Lex Berezhny 1747c9d866 removed --include_reserved from account_list command 2018-10-08 15:02:52 -04:00
Lex Berezhny 1066fbaa7c pylint fix 2018-10-08 15:02:52 -04:00
Lex Berezhny 7f7961f4cc updated integration tests after reducing required confirmations to 0 2018-10-08 15:02:52 -04:00
Lex Berezhny 9d9916548b added channel/signature annotation to all TXOs, support for pagination 2018-10-08 15:02:52 -04:00
Lex Berezhny ada5b55f22 remove unused import 2018-10-08 15:02:52 -04:00
Lex Berezhny a2f28e7788 + permanent_url and TXO annotations 2018-10-08 15:02:52 -04:00
Jack Robison c6092a4c79 bump version 2018-10-08 09:16:54 -04:00
Jack Robison 315a557019 Merge remote-tracking branch 'origin/1439' 2018-10-08 09:15:46 -04:00
Jack Robison 20221e1186 Merge remote-tracking branch 'origin/reduce_routing_table_ping_flood' 2018-10-08 09:15:30 -04:00
Victor Shyba 50003d2600 restore delay on ping queue 2018-10-03 19:21:48 -03:00
Victor Shyba ea6b2b98fb refactor ping queue 2018-10-03 19:21:48 -03:00
Victor Shyba 4890fdfb50 remove semaphore from ping queue 2018-10-03 19:21:48 -03:00
Lex Berezhny 2f95c3a9d1 wallet integration test fixes 2018-10-03 18:06:22 -04:00
Lex Berezhny 6039159ecc use d2l/l2d in integration tests 2018-10-03 18:06:22 -04:00
Lex Berezhny 8e4d8345bd removed unused import 2018-10-03 18:06:22 -04:00
Lex Berezhny 22df26eeb6 dewies_to_lbc and lbc_to_dewies 2018-10-03 18:06:22 -04:00
Lex Berezhny 7b9ff3e8b5 updated with torba refactoring and working claim_send_to_address 2018-10-03 12:32:01 -04:00
Jack Robison 1a5b2c08ee update INSTALL.md 2018-10-02 13:31:29 -04:00
Jack Robison b6babb1c9d bump 2018-10-02 11:07:51 -04:00
Jack Robison a88485c5a0 Merge branch 'fix-missing-wallet-dir' 2018-10-02 11:06:59 -04:00
Jack Robison 0cefa07e97 Merge branch 'fix-1501' 2018-10-02 11:06:35 -04:00
Jack Robison ca9d48d574 fix wallet directory not being created 2018-10-02 10:34:05 -04:00
Jack Robison e10e1de92f [API] fix https://github.com/lbryio/lbry/issues/1501 2018-10-02 10:26:02 -04:00
Jack Robison bdf6887791 Merge branch 'fix-reactor-tests' 2018-10-02 09:09:34 -04:00
Jack Robison 295f2c1115 update INSTALL.md and README.md 2018-10-01 19:42:23 -04:00
Jack Robison 78fcf6a44a add asyncio to requirements 2018-10-01 18:42:28 -04:00
Jack Robison 7dcebf99e0 don't delete the reactor from sys.modules unless in a build 2018-10-01 18:41:45 -04:00
Victor Shyba 463cbde4b1 cast dict_keys to list before using as shortlist 2018-10-01 15:23:50 -03:00
Victor Shyba 1adce8b227 reduce routing table ping flood 2018-10-01 15:17:10 -03:00
Jack Robison 673d259d7a Merge branch '1446' 2018-10-01 08:34:57 -04:00
Jack Robison d0b739a0b9 Merge branch 'fix_peer_ping' 2018-10-01 08:32:30 -04:00
Victor Shyba b4aa51444c fix peer_ping for inexistent peers 2018-10-01 02:09:47 -03:00
Victor Shyba 9178ca701c nullify and refresh tokens 2018-09-28 14:19:51 -03:00
Victor Shyba 7eb9f344f4 test recover from invalid tokens 2018-09-28 13:47:37 -03:00
Victor Shyba 9f8d3e69ae implement get_best_blockhash 2018-09-27 15:30:53 -04:00
Victor Shyba e1130147b5 fix format of resolve on inexistent URI 2018-09-27 15:30:53 -04:00
Jack Robison 0fb6f50aba bump 2018-09-26 20:23:49 -04:00
Jack Robison 91fc772e3c travis 2018-09-26 19:18:53 -04:00
Jack Robison 30b3d31af0 one more 2018-09-26 17:52:21 -04:00
Jack Robison 1a9f81ea14 travis 2018-09-26 17:20:24 -04:00
Jack Robison 5813df74a5 travis 2018-09-26 16:44:01 -04:00
Jack Robison dc6d803fed travis 2018-09-26 15:54:40 -04:00
Jack Robison f9cba69d03 Merge remote-tracking branch 'origin/fix_master' 2018-09-26 15:53:48 -04:00
Victor Shyba 17f9d7cfb2 fix analytics encoding 2018-09-26 16:45:53 -03:00
Jack Robison 42c43e86ef bump version 2018-09-26 14:38:44 -04:00
Jack Robison 8cd230eb6e changelog 2018-09-26 12:23:13 -04:00
Jack Robison dcc71713c3 Merge branch 'fix-1402-1411' 2018-09-26 11:47:42 -04:00
hackrush 3d72b2adf3 Review Fixes 2018-09-26 11:21:32 -04:00
hackrush 655ece18de Changelog update 2018-09-26 11:21:32 -04:00
hackrush cda14f7c69 Deprecated wallet_public_key and claim_renew commands 2018-09-26 11:21:32 -04:00
Jack Robison 3d36e36173 Merge branch 'ssl' 2018-09-26 11:19:48 -04:00
Jack Robison 2dd0221711 [API] support ssl, add use_https setting 2018-09-26 09:56:26 -04:00
Lex Berezhny 1fec9f52ef temporarily return floats until we fix desktop app to recieve string decimals 2018-09-26 07:24:25 -04:00
Lex Berezhny ca9ca09a00 wallet unit test fixes 2018-09-26 07:24:25 -04:00
Lex Berezhny b16c36b55f transaction_list produces same output as old version 2018-09-26 07:24:25 -04:00
Victor Shyba 1c2849235c send startup event right after setup 2018-09-25 17:49:23 -04:00
Jack Robison 202a957a1a Merge branch 'torba-encryption' 2018-09-25 14:59:54 -04:00
Jack Robison 66f33202c7 [API] add account_lock 2018-09-25 13:10:57 -04:00
Jack Robison f310bb6825 [API] fix account_unlock, account_decrypt, and account_encrypt commands 2018-09-25 13:10:55 -04:00
Lex Berezhny 1fb7ffd759 fix integration test to pass amount as decimal string 2018-09-24 01:25:23 -04:00
Mark Beamer Jr 3fcca4bc47 adjusted test to pass string instead of number 2018-09-24 01:25:23 -04:00
Mark Beamer Jr 12c73cf50c adjusted e2e test to use 1.0 instead of 1 per eukreign which came from lyoshenko. 2018-09-24 01:25:23 -04:00
Mark Beamer Jr 02d398613d removed unused imports 2018-09-24 01:25:23 -04:00
Mark Beamer Jr 65e13898c2 fixed pylint 2018-09-24 01:25:23 -04:00
Mark Beamer Jr 559afa465a Implemented removal of float usage. Floats converted to int internally. 2018-09-24 01:25:23 -04:00
Lex Berezhny c753a5825a switched all --account args to --account_id, fixed issues reported by @osilkin98 and regenerated api.json 2018-09-23 23:48:53 -04:00
Victor Shyba 4185a3b9e3 add a pyinstaller hook for coincurve 2018-09-23 22:42:41 -04:00
Lex Berezhny 2f4e6911d6 daemon api json serialization imporovements 2018-09-21 23:24:50 -04:00
Lex Berezhny bcf7218b66 fix integration test 2018-09-21 23:24:50 -04:00
Lex Berezhny e21ade7360 pylint 2018-09-21 23:24:50 -04:00
Lex Berezhny 53d4c149bc added --account to most commands, also claim_list_mine, transaction_list, address_is_mine and other cleanup 2018-09-21 23:24:50 -04:00
Lex Berezhny 0adcaa6a5d utxo_list, transaction_show 2018-09-21 23:24:50 -04:00
Victor Shyba ce4d734250 adds get_claims_for_name 2018-09-21 17:38:07 -04:00
hackrush 04e0933d36 Fixes report bug on python 3 2018-09-21 17:37:01 -04:00
Jack Robison bcf52c3f24 Merge branch 'download_analytics_v2' 2018-09-21 16:51:33 -04:00
Alex Grintsvayg a702afd010 track all downloads in a single db (in parallel with existing metrics) 2018-09-21 15:19:12 -04:00
hackrush ad492a09f7 Added integration tests and other review fixes 2018-09-20 19:45:06 -04:00
hackrush 790991ecad Use account and reduce code + raise exception on claim not found 2018-09-20 19:45:06 -04:00
hackrush f5789775a2 Change sql to use less code 2018-09-20 19:45:06 -04:00
hackrush 5cc0d365c2 Added support to abandon claim using txid and nout 2018-09-20 19:45:06 -04:00
Victor Shyba 5430a0f289 supress git version print statement to debug 2018-09-20 12:49:00 -04:00
Victor Shyba cc7656faa4 adapt content payment calls to new wallet 2018-09-19 10:13:14 -04:00
Victor Shyba 03415415ed improve error message when daemon is already running 2018-09-18 23:09:43 -04:00
Lex Berezhny 1e47ff5a83 argument name changed to avoid redefining variable 2018-09-18 23:08:58 -04:00
Lex Berezhny 88bad1f868 publish accepts channel_id or channel_name 2018-09-18 23:08:58 -04:00
Victor Shyba 91405ef232 explicitly convert page and page_size to int 2018-09-18 22:12:23 -04:00
Jack Robison 570c1163c1 Merge branch 'author-docstring' 2018-09-18 10:12:23 -04:00
Jack Robison 17827036ce update docstring 2018-09-18 10:10:36 -04:00
Jack Robison 338e6c2116 Merge branch 'publish-command-doc-fix' 2018-09-13 12:38:06 -04:00
hackrush cdf6155102 Added which characters are acceptable in the name field of publish command 2018-09-13 12:36:26 -04:00
hackrush e47ef741f7 Review Fixes 2018-09-09 22:00:37 -04:00
hackrush 2878716381 Tests for support and tip 2018-09-09 22:00:37 -04:00
hackrush 6d9947779e Added command to send a tip to a claim id 2018-09-09 22:00:37 -04:00
hackrush c7e5bdd3ae claim_new_support working in the new wallet 2018-09-09 22:00:37 -04:00
Victor Shyba 46dcf8a899 fix wallet reference bug 2018-09-09 21:10:51 -04:00
Victor Shyba c050231ac5 remove file_status flag 2018-09-09 21:10:51 -04:00
Victor Shyba ee5be5adc8 fail graceful when get is called on inexistent URI 2018-09-03 12:48:54 -04:00
hackrush d99ce5f3d6 Integration test for wallet_send (#1395) 2018-09-03 11:41:30 -04:00
Lex Berezhny e7634b74c8 improved test for setting account attributes 2018-09-03 09:09:28 -04:00
Lex Berezhny 26af01a5cc test account list for specific account 2018-09-03 09:09:28 -04:00
Lex Berezhny 36989f1974 removed unnecessary defer.returnValue() 2018-09-03 09:09:28 -04:00
Lex Berezhny 1d40ec34a2 account_balance returns Decimal intead of rounded float 2018-09-03 09:09:28 -04:00
Lex Berezhny f0f30fd75c link to torba repo 2018-09-03 09:09:28 -04:00
Lex Berezhny be2ef747dc deleted commented out code 2018-09-03 09:09:28 -04:00
Lex Berezhny 268a01dc23 run AccountManagement tests on travis 2018-09-03 09:09:28 -04:00
Lex Berezhny 8399f2ab33 account management 2018-09-03 09:09:28 -04:00
Lex Berezhny 7698ab7053 catch InvalidOperation exception when parsing decimals 2018-09-03 09:09:28 -04:00
Lex Berezhny eff66bdf80 use get_dewies_or_error in jsonrpc_publish 2018-09-03 09:09:28 -04:00
Lex Berezhny 70e68c6361 handle amount passed as float 2018-09-03 09:09:28 -04:00
Lex Berezhny 2a53afbfd8 dont fail migrating certs if lbryumx doesnt know about a cert claim 2018-09-03 09:09:28 -04:00
Lex Berezhny a42bf67800 look in all accounts for certificates 2018-09-03 09:09:28 -04:00
Lex Berezhny b783b4daed publish with certificates fix 2018-09-03 09:09:28 -04:00
Lex Berezhny c38b463bf6 added default_wallet and default_account properties to daemon 2018-09-03 09:09:28 -04:00
Lex Berezhny 8696cd3c27 fix for chris45 integration test 2018-09-03 09:09:28 -04:00
Lex Berezhny 0abbc20899 fix for cli test 2018-09-03 09:09:28 -04:00
Lex Berezhny 2fca29144e pylint: line too long fixes 2018-09-03 09:09:28 -04:00
Lex Berezhny 1675cc4580 renamed a bunch of wallet commands and split up the big account command 2018-09-03 09:09:28 -04:00
shyba 93d134f0ce [bug] remove get_block_info as it doesnt make sense, unify to get_block (#1425) 2018-08-31 13:12:13 -04:00
hackrush 1c6ce49b7b Removed unnecessary yield and inlineCallbacks, rectified command behaviour (#1421)
* wallet_prefill_addresses command follows specifications(docs)
* unnecessary to document bugs in changelog which were created and fixed without having ever been released
2018-08-30 15:26:04 -04:00
Jack Robison a7f34e2913 Merge remote-tracking branch 'origin/fix_rerun' 2018-08-28 11:54:33 -04:00
Victor Shyba 2a285d139a bump attempts to 5 and properly check failure 2018-08-27 22:51:16 -03:00
shyba 5e371e8e6a [API] Fixes for block_show and claim_show (#1390)
block_show using block hash
block_show using height
claim_show from claim_id
claim_show from outpoint
2018-08-27 20:03:08 -04:00
Lex Berezhny fdc98fc746 Update README.md
Fixed download links which were pointing to very outdated versions of daemon.
2018-08-26 00:10:38 -04:00
Lex Berezhny 0d8e55bc95 Remove icons directory (#1389)
* removed icons directory, saved 48x48.png icon to be the README.md icon
* removed extra whitespace in lbry.png
* point README to new icon
2018-08-24 17:36:52 -04:00
Lex Berezhny 69851a6588 README and other post lbryum-refactor merge cleanup (#1388)
* updated README with correct Python version, point code coverage to codecov, added header icon and fixed jacks email link
* removed .appveyor.yml, requirements.txt, requirements_testing.txt
2018-08-24 16:44:26 -04:00
Jack Robison b101fafd39 Merge branch 'lbryum-refactor' 2018-08-24 15:04:35 -04:00
Victor Shyba d2aaf7bd09 revert travis to older lbryumx for now 2018-08-24 15:15:57 -03:00
Jack Robison 5a1f551ce5 requirements and bytes/string 2018-08-24 12:21:11 -04:00
Lex Berezhny 584433e16c updated CHANGELOG 2018-08-24 11:42:35 -04:00
Lex Berezhny 0c9c6c2705 create wallet directory if it doesnt exist 2018-08-24 11:42:35 -04:00
Lex Berezhny afae649a9e stubbed out buy/sell claim 2018-08-24 11:42:35 -04:00
Lex Berezhny 2238e1f802 wallet_send command working (#1382)
* wallet_send command
2018-08-24 11:42:35 -04:00
Victor Shyba 518c447fef standardize distance.py to int.from_bytes 2018-08-24 11:42:35 -04:00
Lex Berezhny cb9b348828 WalletComponent expects manager to have is_wallet_unlocked, added a stubbed method for now 2018-08-24 11:42:35 -04:00
shyba eab95a6246 DHT fixes from review and an attempt at removing hashing and equals (#1370)
* use int to_bytes/from_bytes instead of struct
* fix ping queue bug and dht functional tests
* run functional tests on travis
* re-add contact comparison unit test
* dont need __ne__ if its just inverting __eq__ result
2018-08-24 11:42:35 -04:00
Lex Berezhny 593d0046bd Revert "bytes/string"
This reverts commit 594dd61
2018-08-24 11:42:35 -04:00
Lex Berezhny c5d1478c02 changed default maximum number of times an address should get used to 1 from 2, per lbrynaut review 2018-08-24 11:42:35 -04:00
Lex Berezhny 1c57293e9c amount comparison should be 0, per lbrynaut review 2018-08-24 11:42:35 -04:00
hackrush 14a067a4c4 wallet_prefill_command working 2018-08-24 11:42:35 -04:00
hackrush 707ab831fd Fake Analytics for test_cli 2018-08-24 11:42:35 -04:00
hackrush ceb25f917a Use get_dewies_or_error method instead 2018-08-24 11:42:35 -04:00
hackrush 002faf7a58 Convert str to Decimals 2018-08-24 11:42:35 -04:00
Jack Robison d1bbb7af74 update required twisted version 2018-08-24 11:42:35 -04:00
Jack Robison ea603dce13 back up original lbryum wallet before migration
-use same directory and file name for new wallet as the old wallet
-catch error when trying to migrate an abandoned certificate
2018-08-24 11:42:35 -04:00
Jack Robison f2a768c9c4 bytes/string 2018-08-24 11:42:35 -04:00
Jack Robison 12384a89a1 remove wsgiref from requirements 2018-08-24 11:42:35 -04:00
Jack Robison c21956b4f3 handle distro import error 2018-08-24 11:42:35 -04:00
Lex Berezhny 4a8776be10 wallet integration tests fixed 2018-08-24 11:42:35 -04:00
Lex Berezhny d6d69a46d4 added missing yield 2018-08-24 11:42:35 -04:00
Jack Robison 4384a5f022 fix json encoding of bytes 2018-08-24 11:42:35 -04:00
hackrush aa83bbd637 sq close the goddamn session 2018-08-24 11:42:35 -04:00
hackrush daae0b2836 sq return not yield, run on clear field 2018-08-24 11:42:35 -04:00
Lex Berezhny 5afd446bb3 fixing wallet unit tests test_ledger.py and test_transactions.py 2018-08-24 11:42:35 -04:00
hackrush 2b5e3204c0 Refactor client, cli and test_cli, fix delayedCalls not expiring 2018-08-24 11:42:35 -04:00
Lex Berezhny 84c91c480f fix for unit/wallet/test_accounts.py 2018-08-24 11:42:35 -04:00
Lex Berezhny cbc98e72a7 pylint fixes 2018-08-24 11:42:35 -04:00
Lex Berezhny d1f00255c1 test fixes related to headers 2018-08-24 11:42:35 -04:00
Lex Berezhny 434d98c6a6 restore formatting 2018-08-24 11:42:35 -04:00
Lex Berezhny e51c98ca8c formatting fixes 2018-08-24 11:42:35 -04:00
Lex Berezhny f719aa1db0 old command is called daemon_stop 2018-08-24 11:42:35 -04:00
Lex Berezhny 3686b1d970 changes from jacks review 2018-08-24 11:42:35 -04:00
Lex Berezhny e6fd8cc0f2 wip new header stuff 2018-08-24 11:42:35 -04:00
Lex Berezhny 8739dc5d2c pin faker to 0.8.17 2018-08-24 11:42:33 -04:00
hackrush d6c04d3586 sq Fix tests 2018-08-24 11:42:08 -04:00
hackrush 7236203f1e sq Add new integration tests to tox.ini 2018-08-24 11:42:08 -04:00
hackrush 8c6c442fdd Fixed regression in unauthenticated API client w/ integration tests 2018-08-24 11:42:08 -04:00
Lex Berezhny 49c659b832 assert integration test is actually using authentication 2018-08-24 11:42:08 -04:00
hackrush 2590523fa1 Rename client and add test_cli to tox.ini 2018-08-24 11:42:08 -04:00
hackrush 08b4fd6942 stop listening port only if it is not None 2018-08-24 11:42:08 -04:00
hackrush 6a8963d807 Added Authenticated API Client Integration Test 2018-08-24 11:42:08 -04:00
hackrush 083bf4f61d aiohttp based Async Authenticated API Client 2018-08-24 11:42:08 -04:00
Lex Berezhny ab3b0134b5 Purchase Transaction Type 2018-08-24 11:42:08 -04:00
Victor Shyba 3b88d2465d fixes and refactors from review 2018-08-24 11:42:08 -04:00
Victor Shyba 0841c90e6c fix encoding on download 2018-08-24 11:42:06 -04:00
Victor Shyba 3f6e928cc2 fix findValue result parsing 2018-08-24 11:40:43 -04:00
Victor Shyba a32a0c6401 fix peer_ping 2018-08-24 11:40:43 -04:00
Victor Shyba 95cb29d3c8 fix routing_table_get 2018-08-24 11:40:43 -04:00
Victor Shyba e85f8b245b fix conf.py 2018-08-24 11:40:43 -04:00
Victor Shyba 04836ea0d9 fixes from review 2018-08-24 11:40:43 -04:00
Victor Shyba 451823f33e test_misc and test_streamify functionals on py3 2018-08-24 11:40:43 -04:00
Victor Shyba 8c3552dd34 port reflector 2018-08-24 11:40:43 -04:00
Lex Berezhny a204f0d3e6 - replaced old lbryum function with struct.pack, per @BrannonKing review
- make copy of keys so we can modify the dictionary
2018-08-24 11:40:43 -04:00
Lex Berezhny 892758be65 account.from_dict() takes wallet as second arg 2018-08-24 11:40:43 -04:00
Lex Berezhny 3594c8976d improved output when migrating certificates 2018-08-24 11:40:43 -04:00
Lex Berezhny 23ede44bff fix blob downloader test on py3
not tying lbrynet db to wallet db just yet
2018-08-24 11:40:43 -04:00
Lex Berezhny 0006a68bf2 fix import in blob downloder tests 2018-08-24 11:40:43 -04:00
Lex Berezhny 0349e68201 moved unit tests into unit test directory 2018-08-24 11:40:43 -04:00
Lex Berezhny be248dc448 account command can now configure the default account 2018-08-24 11:40:43 -04:00
Lex Berezhny d0d5d0340d unit tests fixed 2018-08-24 11:40:43 -04:00
Lex Berezhny f79a49bbf4 added account management command 2018-08-24 11:40:43 -04:00
Lex Berezhny 6a5d88a0d5 new fund command and automatic account creation 2018-08-24 11:40:43 -04:00
hackrush 4165881d5f Authenticated CLI works again 2018-08-24 11:40:43 -04:00
Lex Berezhny 8dc4e3be43 integration tests working again and daemon starts normally 2018-08-24 11:40:43 -04:00
Lex Berezhny 0badea874d test_Downloder tests fixed 2018-08-24 11:40:43 -04:00
Lex Berezhny 0dd6193eaa FakeComponent was missing __lt__ comparison operator and couldnt be put in set() 2018-08-24 11:40:43 -04:00
Lex Berezhny d10778430f upnpclient -> txupnp 2018-08-24 11:40:43 -04:00
Lex Berezhny da8e09846d pylint fixed 2018-08-24 11:40:43 -04:00
Lex Berezhny 310fe4a42c updated Headers component to use lbrynet.wallet 2018-08-24 11:40:43 -04:00
hackrush a7ef8889dd Unified CLI, python 3(WIP) (#1330)
* Added new custom cli class using aiohttp
* Proper error handling in CLI based on RPC error codes(PoC)
* Auth API working
* UnitTests
2018-08-24 11:40:43 -04:00
Lex Berezhny f41229cb5b tx.get_claim_id() -> txo.claim_id, claim update works now
channel_list encodes claim name and also added claim_id
fixed BlobManager foreign key error handling
2018-08-24 11:40:43 -04:00
Lex Berezhny 8ab4e3ca49 + channel_list command works again but test_commands.py integration test is failing 2018-08-24 11:40:43 -04:00
Victor Shyba ff8d37443e more checks on test_transactions 2018-08-24 11:40:43 -04:00
Lex Berezhny fcd46629c4 refactored how transactions are created, fixed list addresses command 2018-08-24 11:40:43 -04:00
Victor Shyba 9ad9eb083b fix integration test NBO encoding 2018-08-24 11:40:43 -04:00
Lex Berezhny 4669507880 Transaction.abandon() now requires list of outputs 2018-08-24 11:40:43 -04:00
Lex Berezhny 9348f4f366 add missing new line at end of file 2018-08-24 11:40:43 -04:00
Lex Berezhny 47bb634035 abandon claims and chris45 epic adventure 2018-08-24 11:40:43 -04:00
Lex Berezhny 10b34d6b33 unnecessary list() added during py3 port
instead of recursive bytes2unicode use a proper JSONEncoder to conver bytes->unicode for json.dumps()
removing excessive isinstance(data, bytes) checks
py3: / -> // and list() around .items() that gets modified in loop
moved lbrynet.undecorated to where its actually used and hopefully we can delete it eventually
removed build/upload_assets.py, travis can do all this now
2018-08-24 11:40:43 -04:00
Lex Berezhny f061ca2b15 integration tests no longer mock Blob, BlobManager and Session 2018-08-24 11:40:43 -04:00
Lex Berezhny bc24dbea29 refactoring of DHT tests and fixed encoding bug when dealing with bytearray 2018-08-24 11:40:43 -04:00
Lex Berezhny 2d4bf73632 working on unit tests 2018-08-24 11:40:43 -04:00
Lex Berezhny c21d8cc8dc update wallet migration 2018-08-24 11:40:43 -04:00
Lex Berezhny cc6a91a477 windows travis build is lagnuage generic 2018-08-24 11:40:43 -04:00
Lex Berezhny 21a2725215 travis 2018-08-24 11:40:43 -04:00
Lex Berezhny 411b8c74cc travis yaml conslidation via anchors 2018-08-24 11:40:43 -04:00
Lex Berezhny 91bf312e4d fix for file open during publish 2018-08-24 11:40:43 -04:00
Lex Berezhny 277b8b122c run integration tests with 3.7 2018-08-24 11:40:43 -04:00
Lex Berezhny 59c49d2d31 dont run DHT tests, until we can make them run faster 2018-08-24 11:40:43 -04:00
Lex Berezhny 16024c95aa py2->py3 unhexlify() 2018-08-24 11:40:43 -04:00
Lex Berezhny 923ed80fcf run DHT tests as their own job 2018-08-24 11:40:43 -04:00
Lex Berezhny 2ee2916f78 run functional DHT tests on travis 2018-08-24 11:40:43 -04:00
Lex Berezhny 79bf1b427f work around for a bug in travis artifact uploader 2018-08-24 11:40:43 -04:00
Lex Berezhny c21c329aa0 debug artifact uploading 2018-08-24 11:40:43 -04:00
Lex Berezhny 6b3af4bc21 fix for travis buid to put build binary in right place 2018-08-24 11:40:43 -04:00
Lex Berezhny ffbdf03847 switched from testing on 3.7 to 3.6 2018-08-24 11:40:43 -04:00
Lex Berezhny 8d08bfb3f9 moved build/icons up one level and deleted build directory 2018-08-24 11:40:43 -04:00
hackrush 1af6326b97 Fix account_balance's doc strings 2018-08-24 11:40:43 -04:00
Victor Shyba 31630a84be reduce entropy on DHT test suite 2018-08-24 11:40:43 -04:00
hackrush 0601bf3197 sq Learning better coding practices with Lex 2 2018-08-24 11:40:43 -04:00
hackrush 826d2866fa sq Learning better coding practices with Lex 2018-08-24 11:40:43 -04:00
hackrush ba00498d83 Make commands work from terminal 2018-08-24 11:40:43 -04:00
Lex Berezhny 4da40e8d52 renamed build binary to lbrynet and other cleanup 2018-08-24 11:40:41 -04:00
Lex Berezhny d54d989c40 change stop command from daemon_stop to just stop lbrynet 2018-08-24 11:38:20 -04:00
Lex Berezhny 18ecbba87c fix cli formatting 2018-08-24 11:38:20 -04:00
Lex Berezhny 9b417486a9 altogether now! 2018-08-24 11:38:20 -04:00
Lex Berezhny 6d208ac496 trying pip3 on mac 2018-08-24 11:38:20 -04:00
Lex Berezhny 8990c145fb python already installed on mac 2018-08-24 11:38:20 -04:00
Lex Berezhny e5b1c3c829 dont manually install python on mac 2018-08-24 11:38:20 -04:00
Lex Berezhny 6c7128c0f6 trying mac build 2018-08-24 11:38:20 -04:00
Lex Berezhny f31f1fa40b switching down to python 3.6 until pyinstaller and wine support 3.7 2018-08-24 11:38:20 -04:00
Lex Berezhny 2ceba79b9f linux build added and uploading to S3 2018-08-24 11:38:19 -04:00
Lex Berezhny 1c8a59c1a9 upgrade upload assets script to py3 2018-08-24 11:37:53 -04:00
Lex Berezhny 298bf91293 moved build/upload_assets.py -> scripts/upload_assets.py 2018-08-24 11:37:53 -04:00
Victor Shyba b25d592d99 more dht fixes and most of functional tests 2018-08-24 11:37:53 -04:00
Lex Berezhny 875edb5f76 + aiohttp dependency added 2018-08-24 11:37:53 -04:00
Lex Berezhny 84d97ab323 + account_max_gap command 2018-08-24 11:37:53 -04:00
Lex Berezhny aeaffd620e simple cli implementation 2018-08-24 11:37:53 -04:00
hackrush 3f7091cbf2 Make curl work in py3 again 2018-08-24 11:37:53 -04:00
hackrush a95f791215 Make daemon work in py3 again 2018-08-24 11:37:53 -04:00
Lex Berezhny af2aeaa66b fixing unit tests 2018-08-24 11:37:53 -04:00
Lex Berezhny 855fd8bf9a moved test_customLogger.py 2018-08-24 11:37:53 -04:00
Lex Berezhny a937aff80f pylint fixes 2018-08-24 11:37:53 -04:00
Lex Berezhny 5597d45aed got integration tests to work again with rebased branch 2018-08-24 11:37:53 -04:00
hackrush 473d2eabfa curl works with python3 now 2018-08-24 11:37:53 -04:00
Victor Shyba 78c8c8e64d more porting, plus some functional tests working 2018-08-24 11:37:53 -04:00
Lex Berezhny 7335d012ef replace miniupnpc with upnpclient
docker build with wine
2018-08-24 11:37:53 -04:00
Victor Shyba 99be38604a attempt to fix tests isolation 2018-08-24 11:37:53 -04:00
Lex Berezhny d35d340613 fixing unit tests
fixing integration tests
skip running functional tests on travis until they are fixed
2018-08-24 11:37:53 -04:00
Lex Berezhny f05ca137be adding build stages to .travis.yml 2018-08-24 11:37:53 -04:00
Lex Berezhny e718caca77 final pylint fixes (hopefuly) 2018-08-24 11:37:53 -04:00
Lex Berezhny 693a3346d2 pylint fixes 2018-08-24 11:37:53 -04:00
Lex Berezhny 1f75b6a49e pylint skips assignment-from-no-return, useless-return 2018-08-24 11:37:53 -04:00
Lex Berezhny 4ece422f48 No longer inheriting from object and added proper use of super(). 2018-08-24 11:37:53 -04:00
Lex Berezhny 607677f98f iterkeys() -> keys() 2018-08-24 11:37:53 -04:00
Victor Shyba e1e7be63b8 more fixes on dht functionals 2018-08-24 11:37:53 -04:00
Victor Shyba e1314a9d1e working functional test_contact_rpc + more string bans 2018-08-24 11:37:53 -04:00
Victor Shyba 1ee682f06f make bencoding right for both py3+py2 2018-08-24 11:37:53 -04:00
Lex Berezhny ba80c0e594 py2(iteritems, itervalues) -> py3(items, values) 2018-08-24 11:37:53 -04:00
Lex Berezhny fbdbcc8070 assertEquals() -> assertEqual() 2018-08-24 11:37:53 -04:00
Lex Berezhny 24a872885a xrange() -> range() 2018-08-24 11:37:53 -04:00
Lex Berezhny 005a131440 require python 3.7 2018-08-24 11:37:53 -04:00
Lex Berezhny 0b1cb16353 windows related fixes, in py3 everything is unicode, no need to convert paths to bytes 2018-08-24 11:37:53 -04:00
Lex Berezhny 1a3eaf7e6a unhexlify/hexlify some values in database/storage.py 2018-08-24 11:37:53 -04:00
Lex Berezhny 971252d5d1 undo simplejson commit 2018-08-24 11:37:53 -04:00
Lex Berezhny c5ba9b263e use twisted[tls] instead of specifying twisted dependencies in our own setup.py 2018-08-24 11:37:53 -04:00
hackrush c01716a6c0 Using simplejson and fixed some encodes and decodes 2018-08-24 11:37:53 -04:00
hackrush 0fbf131df3 default certificates is a dict now 2018-08-24 11:37:53 -04:00
hackrush 4bd0806e58 Fix metaclass not registering objects 2018-08-24 11:37:53 -04:00
hackrush 4cbb71705e Added pyopenssl and service-identity dependency to setup.py 2018-08-24 11:37:53 -04:00
Lex Berezhny 8ab4dd6b2c build py37, using ubuntu xenail on traivs 2018-08-24 11:37:53 -04:00
Lex Berezhny 318c369752 made tests into a package again and moved mocks.py back to root tests dir 2018-08-24 11:37:53 -04:00
Victor Shyba cea3b7630c partial dht functional tests porting to py3 2018-08-24 11:37:53 -04:00
Victor Shyba 19211d4417 make sure bencoding works for bytes, not strings 2018-08-24 11:37:53 -04:00
Victor Shyba c312d1b3a6 all unit/dht green on py3 2018-08-24 11:37:53 -04:00
Victor Shyba e4ea1ccbfb test_node green on py3 2018-08-24 11:37:53 -04:00
Victor Shyba 5b35c4e8f0 port dht.distance ~> py3 2018-08-24 11:37:53 -04:00
Victor Shyba 9967857a57 port dht.encoding ~> py3 2018-08-24 11:37:53 -04:00
Victor Shyba 78c560a3be general incomplete porting of daemon booting ~> py3 2018-08-24 11:37:53 -04:00
Victor Shyba 07a12b66e8 port dht.contacts ~> py3 2018-08-24 11:37:53 -04:00
Lex Berezhny a28c9d09c8 - fix and improvements related to the two balance command account_balance and wallet_balance
- working CommonWorkflowTests integration test
- pylint, unit and integration test fixes
- switch integration tests to use async/await
2018-08-24 11:37:53 -04:00
Victor Shyba b2ab13187f fix get_certificates 2018-08-24 11:37:53 -04:00
Victor Shyba 5ec696fbfd fix lookup helper by not calling hexlify twice 2018-08-24 11:37:53 -04:00
Victor Shyba 44bd0ae847 fix another forgotten claim id nbo conversion 2018-08-24 11:37:53 -04:00
Victor Shyba 5809ba093d convert amount on channel_new 2018-08-24 11:37:53 -04:00
Lex Berezhny 17aee92dc9 wallet_balance --include-unconfirmed works now 2018-08-24 11:37:53 -04:00
Victor Shyba 6f1d2e7690 fix RPC publish claim serialization 2018-08-24 11:37:53 -04:00
Victor Shyba 75f57fc517 fix claim id handling during publish from the latest changes 2018-08-24 11:37:53 -04:00
Lex Berezhny f589410496 tests 2018-08-24 11:37:53 -04:00
Lex Berezhny aecc7c6643 resolve works in py3!!!!!11111oneoneone 2018-08-24 11:37:53 -04:00
Victor Shyba c5c1b62939 fix resolve from tx.id rename 2018-08-24 11:37:53 -04:00
Lex Berezhny adc92a2b52 fix for daemon.get_est_cost 2018-08-24 11:37:53 -04:00
Lex Berezhny 9636ca22e7 fixing unit tests 2018-08-24 11:37:53 -04:00
Lex Berezhny 0442770c9d pylint fixes 2018-08-24 11:37:53 -04:00
Lex Berezhny 3b6a879fc9 add is_hd=true for default migrated wallet 2018-08-24 11:37:53 -04:00
Lex Berezhny a1b4c9acd6 claim_id is hexlified before saving to sqlite 2018-08-24 11:37:53 -04:00
Lex Berezhny 0f90dee224 faililng test 2018-08-24 11:37:53 -04:00
Lex Berezhny 159e84468c py3 support in resolve 2018-08-24 11:37:53 -04:00
Lex Berezhny 59e4ac30c2 fixing py2 stuff that broke during py3 fixes 2018-08-24 11:37:53 -04:00
Lex Berezhny b5ce948bc1 convert buffer to string 2018-08-24 11:37:53 -04:00
Lex Berezhny 076af7ef43 py3 compatibility updates 2018-08-24 11:37:53 -04:00
Lex Berezhny 43bef9447c progress on publish command: py3 porting and integration tests 2018-08-24 11:37:53 -04:00
Victor Shyba b1c5fe0b4d cryptblob write needs bytes on py3 2018-08-24 11:37:53 -04:00
Victor Shyba 39d7f2e46e port cryptblob test to py3 2018-08-24 11:37:53 -04:00
Lex Berezhny 315661208d integration test publishes actual file instead of just sources 2018-08-24 11:37:53 -04:00
Lex Berezhny e175e43082 .decode() string for sqlite 2018-08-24 11:37:53 -04:00
Lex Berezhny cb9b320ed7 dropped make_claim in favor of wallet.claim_name 2018-08-24 11:37:53 -04:00
Lex Berezhny 1d906fb0c7 create_and_publish_stream fix 2018-08-24 11:37:53 -04:00
Lex Berezhny c544e26206 publish integration test works 2018-08-24 11:37:53 -04:00
Lex Berezhny 1eaac35a3e claim_name does not take change_address anymore 2018-08-24 11:37:53 -04:00
Lex Berezhny cc7962fc2e stringify address from sqlite 2018-08-24 11:37:53 -04:00
Lex Berezhny 3638984d67 fixed getting unused address 2018-08-24 11:37:53 -04:00
Lex Berezhny 62e77c69f5 jsonrpc_publish takes amount in lbc, not satoshi 2018-08-24 11:37:53 -04:00
Lex Berezhny 774d813576 fixed encoding issue on saving wallet 2018-08-24 11:37:53 -04:00
Lex Berezhny 688caf4453 + save wallet after creating a channel 2018-08-24 11:37:53 -04:00
Lex Berezhny eeb5b3c139 publishing 2018-08-24 11:37:53 -04:00
Lex Berezhny b5e6dd1060 + maybe_migrate_certificates 2018-08-24 11:37:53 -04:00
Lex Berezhny 272d818050 saving txo bug fix 2018-08-24 11:37:53 -04:00
Lex Berezhny 0fc1bd7a8c store claim_id 2018-08-24 11:37:53 -04:00
Lex Berezhny 2b234e0713 fix syntax error 2018-08-24 11:37:53 -04:00
Victor Shyba 9e8cb17ecb insane working version of paged resolve, to be refactored 2018-08-24 11:37:53 -04:00
Victor Shyba e0230864ce creates a Resolve class for moving resolve methods out of ledger 2018-08-24 11:37:53 -04:00
Victor Shyba 9e89b3e40e start a new module for isolating resolve related code before we can refactor it 2018-08-24 11:37:53 -04:00
Victor Shyba 036663ae62 adds get_claims_by_ids and fix a test name 2018-08-24 11:37:53 -04:00
Victor Shyba b62321689d test proofs 2018-08-24 11:37:53 -04:00
Victor Shyba bdd271e78f adds resolve 2018-08-24 11:37:53 -04:00
Lex Berezhny 4a56d38782 certificate.py 2018-08-24 11:37:53 -04:00
Lex Berezhny 70a7ca95fe wip 2018-08-24 11:37:53 -04:00
Lex Berezhny f85e61d8ed run functional tests first
run two integration tests on travis 😱
reduce integration test verbosity
2018-08-24 11:37:53 -04:00
Lex Berezhny ac8c641d64 use lbryumx branch of electrumx 2018-08-24 11:37:53 -04:00
Lex Berezhny d81502e191 unit test fixes, balance/utxos filters out claims, abandoning claims 2018-08-24 11:37:53 -04:00
Lex Berezhny 8c35d355e5 added is_claim,is_support,is_update and claim_name attributes to txo able 2018-08-24 11:37:53 -04:00
Lex Berezhny 72b004664f config fix 2018-08-24 11:37:53 -04:00
Lex Berezhny 49a7408968 travis-ci stuff: only run pylint for py27 in tox and checkout lbryschema 2018-08-24 11:37:53 -04:00
Lex Berezhny 17738ad248 import fixes due to tests directory moving 2018-08-24 11:37:53 -04:00
Lex Berezhny 425c952293 added tox 2018-08-24 11:37:53 -04:00
Lex Berezhny 03d2d0e237 fixed import paths as a result of moving the tests 2018-08-24 11:37:53 -04:00
Lex Berezhny edfc2c6124 fix to make sd_hash more deterministic 2018-08-24 11:37:53 -04:00
Lex Berezhny 545451a829 moved ./lbry/tests to ./tests 2018-08-24 11:37:53 -04:00
Lex Berezhny 12ff7015cd wallet tests and py3 compatibility 2018-08-24 11:37:53 -04:00
Lex Berezhny f2f212edba switch component wallet to use the new wallet 2018-08-24 11:37:53 -04:00
Lex Berezhny 69446491b8 pylint and unit test fixes 2018-08-24 11:37:50 -04:00
Lex Berezhny d5beaa0937 jsonrpc_channel_new uses new wallet 2018-08-24 11:37:25 -04:00
Lex Berezhny cc970c499b daemon/auth py3 conversion, this breaks the client for now 2018-08-24 11:37:25 -04:00
Lex Berezhny 0087b16912 deleted old broken test 2018-08-24 11:37:25 -04:00
Lex Berezhny 5520d518b5 DHT py3 compatibility, mostly commenting out implements() and fixing imports
cryptstream py3 support, mostly commenting out implements()
lbry_file py3 support, mostly commenting out implements()
file_manager py3 support, mostly commenting out implements()
core py3 support, mostly commenting out implements() and fixing imports
2018-08-24 11:37:25 -04:00
Lex Berezhny b3e9240aa8 wip, valuesforuris 2018-08-24 11:37:25 -04:00
Lex Berezhny 137a270488 * channel certificate creation and taking advantage of is_reserved feature on txos to exclude output from being used in new transactions 2018-08-24 11:37:25 -04:00
Lex Berezhny e36610d63c fix resolve 2018-08-24 11:37:25 -04:00
Lex Berezhny c4273fbef0 claim test 2018-08-24 11:37:25 -04:00
Lex Berezhny a4e9f3100f updating wallet tests 2018-08-24 11:37:25 -04:00
Lex Berezhny 2668ceac0e updated to newer torba 2018-08-24 11:37:25 -04:00
Lex Berezhny fb2fb5c38c recovered an old stash 2018-08-24 11:37:25 -04:00
Lex Berezhny 56175df121 wip 2018-08-24 11:37:25 -04:00
Lex Berezhny 7d3daa9fe6 set the lbryschema BLOCKCHAIN_NAME global variable 2018-08-24 11:37:25 -04:00
Lex Berezhny 1d2de27536 import fix 2018-08-24 11:37:25 -04:00
Lex Berezhny aab42e56bd removing lbryum 2018-08-24 11:37:25 -04:00
Lex Berezhny 1368a6e074 wallet manager refactoring
test uses manager.send_amount_to_address instead of manually creating tx
2018-08-24 11:37:25 -04:00
Lex Berezhny 80990b959c moved files around 2018-08-24 11:37:25 -04:00
Lex Berezhny 053b6d1c80 removing lbryum from requirements 2018-08-24 11:37:25 -04:00
Lex Berezhny 692d8e8e1a new wallet becoming drop-in replacement for old lbryum wallet 2018-08-24 11:37:25 -04:00
Lex Berezhny 5e71dcbaf0 wip: implementation is now generic and supports multiple currencies 2018-08-24 11:37:25 -04:00
Lex Berezhny 83958604d5 finally generating fully signed tx submittable to lbrycrd 2018-08-24 11:37:25 -04:00
Lex Berezhny dcd8a6bb0e branch and bound based coin selection with random draw fallback 2018-08-24 11:37:25 -04:00
Lex Berezhny 7161a0edb9 new bitcoin script parser 2018-08-24 11:37:25 -04:00
Lex Berezhny 0fd160e6e6 a lot more stuff 2018-08-24 11:37:25 -04:00
Lex Berezhny ca8b2dd83e wip: initial import of twisted based wallet 2018-08-24 11:37:25 -04:00
Jack Robison 79d1da5ff8 Bump version 0.21.2rc3 --> 0.21.2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-23 23:43:35 -04:00
Jack Robison d73252b11a Bump version 0.21.2rc2 --> 0.21.2rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-23 22:40:16 -04:00
Jack Robison 1f5c56cac3 Merge branch 'fix-mirror-lc' 2018-08-23 22:40:04 -04:00
Jack Robison 18db2aa970 changelog 2018-08-23 22:38:55 -04:00
Jack Robison a7a53f9381 fix mirror downloader looping call 2018-08-23 22:36:30 -04:00
Jack Robison f4b2a05fff fix a typo 2018-08-23 18:35:47 -04:00
Jack Robison cdd2b6bfc1 Bump version 0.21.2rc1 --> 0.21.2rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-23 11:57:49 -04:00
Jack Robison 26bd27cd32 Bump version 0.21.1 --> 0.21.2rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-23 11:45:07 -04:00
Jack Robison 613feb0808 Merge branch 'fix-mirror-download-of-partial-hosted' 2018-08-23 11:44:48 -04:00
Jack Robison d4849443e5 changelog 2018-08-23 11:22:03 -04:00
Jack Robison db70f7e98a requirements
-bump txupnp requirement
-set Faker requirement
2018-08-23 11:21:53 -04:00
Jack Robison 37ddaeca12 fix ping queue bug 2018-08-23 11:18:28 -04:00
Jack Robison 2c546b2cde retry mirror downloads 2018-08-23 11:18:28 -04:00
Jack Robison c48d195951 docs 2018-08-22 00:20:30 -04:00
Alex Grintsvayg 149fa625e0 back to reflector 1 2018-08-15 16:11:34 -04:00
Jack Robison 16cda60f31 fix test 2018-08-13 17:16:48 -04:00
Jack Robison 0d54f6215f Bump version 0.21.1rc2 --> 0.21.1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-13 16:20:16 -04:00
Jack Robison a9774a293f docs 2018-08-13 16:18:31 -04:00
Jack Robison ee8cf08c0e Bump version 0.21.1rc1 --> 0.21.1rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-13 14:04:39 -04:00
Jack Robison ea481e6c5a fix download_progress in blockchain_headers status
-use a 0-100 range for percentage regardless of how much we have to download (if we're resuming or starting fresh)
-prevent negative blocks or blocks_behind counts
2018-08-13 14:03:15 -04:00
Jack Robison 36df9e8283 Bump version 0.21.0 --> 0.21.1rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-12 22:16:00 -04:00
Jack Robison 0467b65e69 docs 2018-08-12 22:15:43 -04:00
Jack Robison 68f1dda7b0 initialize headers download progress 2018-08-12 22:05:07 -04:00
Jack Robison 5b7f7c87bf fix stop_lbry_files 2018-08-12 22:04:55 -04:00
Jack Robison 153022a1a7 add is_locked to status 2018-08-12 22:04:48 -04:00
Lex Berezhny 1098ca0494 + convert float to decimal for wallet_send 2018-08-09 19:14:56 -04:00
Jack Robison d4a7dbf805 Bump version 0.21.0rc13 --> 0.21.0
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-09 13:44:28 -04:00
Jack Robison 56ee815eec Bump version 0.21.0rc12 --> 0.21.0rc13
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-09 13:20:15 -04:00
Jack Robison fe43764bcb fix announcement of blobs downloaded from a http mirror 2018-08-09 13:19:59 -04:00
Jack Robison 93be8b9e31 more changelog cleanup 2018-08-09 12:18:25 -04:00
Jack Robison 2448d51ed5 Bump version 0.21.0rc11 --> 0.21.0rc12
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-09 12:07:35 -04:00
Jack Robison 2804fa4448 fix disabling the hash announcer 2018-08-09 12:04:14 -04:00
Jack Robison 90a119bb2c changelog 2018-08-09 11:56:28 -04:00
Jack Robison 69ae60b4bb Bump version 0.21.0rc10 --> 0.21.0rc11
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-09 11:29:49 -04:00
Jack Robison dca4af942f also try/except around upnp.discover
-catches edge case where miniupnpc fallback is used and a device is found but selectigd fails
2018-08-09 11:28:07 -04:00
Jack Robison 2b7e135839 fix error in blob_delete 2018-08-09 11:26:57 -04:00
Jack Robison d565aa0b3f Bump version 0.21.0rc9 --> 0.21.0rc10
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-09 09:24:39 -04:00
Jack Robison 8b53140581 set downloading_headers to False after the download finishes or fails 2018-08-09 09:24:04 -04:00
Jack Robison 917bae568d fix disabling blob announcement by setting concurrent announcers to 0 2018-08-09 09:22:55 -04:00
Jack Robison efb6af4d51 Bump version 0.21.0rc8 --> 0.21.0rc9
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-09 09:01:57 -04:00
Jack Robison 2361d8bdc1 clean up changelog 2018-08-09 09:01:33 -04:00
Jack Robison 5fe8022b15 don't attempt to announce blobs until we we have contacts in routing 2018-08-09 09:00:36 -04:00
Jack Robison 7931397595 Bump version 0.21.0rc7 --> 0.21.0rc8
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-09 08:44:23 -04:00
Jack Robison 0501f52805 fix upnp edge case
-fixes https://github.com/lbryio/lbry/issues/1352
2018-08-09 08:42:51 -04:00
Jack Robison 108d2a033b fix attribute error when reflector component is disabled 2018-08-09 08:32:43 -04:00
Jack Robison 17a9cfd65c Bump version 0.21.0rc6 --> 0.21.0rc7
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-08 14:12:01 -04:00
Jack Robison 7069ffa37d fix pip 2018-08-08 14:09:33 -04:00
Jack Robison 8eb740cf67 Bump version 0.21.0rc5 --> 0.21.0rc6
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-08 13:58:17 -04:00
Jack Robison 7803eb2e5a bump txupnp 2018-08-08 13:57:41 -04:00
Jack Robison b73e50e64b Bump version 0.21.0rc4 --> 0.21.0rc5
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-07 19:27:50 -04:00
Jack Robison c73148a1fb Merge remote-tracking branch 'origin/fix_1346' 2018-08-07 19:27:28 -04:00
Jack Robison 65a02190dc Merge remote-tracking branch 'origin/fix_1344' 2018-08-07 19:26:15 -04:00
Jack Robison 74b49fe69f Merge branch 'upnp-fixes' 2018-08-07 19:25:06 -04:00
Jack Robison 26e32472d5 fix upnp and peer port setup
-bump txupnp requirement
2018-08-07 19:04:54 -04:00
Victor Shyba 20d8b2963c adds a CHANGELOG 2018-08-07 14:01:46 -03:00
Victor Shyba d21313e590 allow mirrors if conf is set, but clear it for tests 2018-08-07 13:55:45 -03:00
Victor Shyba 81e9c9bf74 adds a CHANGELOG 2018-08-07 13:53:15 -03:00
Victor Shyba 382226ff9a tells that an address is wrong earlier on publishing 2018-08-07 13:52:01 -03:00
Jack Robison 9511040d5a fix upnp and peer port setup 2018-08-05 13:12:39 -04:00
Jack Robison 049936f4a6 docs 2018-08-03 13:32:57 -04:00
Jack Robison 15161dd429 fix docstring 2018-08-03 13:31:51 -04:00
Jack Robison efc5982860 Bump version 0.21.0rc3 --> 0.21.0rc4
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-08-03 13:22:06 -04:00
Jack Robison acb95fdcdf Merge remote-tracking branch 'origin/greedy_search' 2018-08-03 13:21:29 -04:00
Jack Robison f1b19f10cc Merge branch 'master' into greedy_search 2018-08-03 13:08:31 -04:00
Jack Robison 6eaf1ac2a0 Merge branch 'update-cryptography' 2018-08-03 12:52:47 -04:00
Jack Robison 01ee1e1468 Merge branch 'component-status' 2018-08-03 12:51:49 -04:00
Jack Robison 83dc295b7b update cryptography requirement 2018-08-03 12:38:28 -04:00
Jack Robison 4a28c620a4 changelog 2018-08-03 12:35:13 -04:00
Jack Robison e50ac69358 tests and pylint 2018-08-03 12:35:10 -04:00
Jack Robison aef561d78b refactor status results 2018-08-03 12:33:31 -04:00
Jack Robison 4ef4a4816f update component tests 2018-08-03 11:37:10 -04:00
Jack Robison 8533bfa7c5 refactor test_misc, remove Session
-use only the one process
2018-08-03 11:37:10 -04:00
Jack Robison 6389e961b9 update (and refactor) tests to not use Session 2018-08-03 11:37:10 -04:00
Jack Robison 6359fc608e set download_mirrors upon making the downloader instead in the file manager globally 2018-08-03 11:37:10 -04:00
Jack Robison 314400a1bd move looping call to check announcement status to SQLiteStorage 2018-08-03 11:37:10 -04:00
Jack Robison a3de065c93 use txupnp instead of miniupnpc 2018-08-03 11:37:07 -04:00
Jack Robison 99207b7221 delete Session.py
split Session into Components
2018-07-30 18:00:33 -04:00
Jack Robison e3c3fafa1e split headers download into own component
-add component statuses
2018-07-30 18:00:33 -04:00
Jack Robison 5b3103e41b Merge pull request #1315 from lbryio/mirroring_refactor
refactor mirroring
2018-07-30 16:00:02 -04:00
Victor Shyba 9742da348b do not consume errors on DeferredList 2018-07-30 13:53:58 -03:00
Victor Shyba 94a31922f7 fix unnecessary logging tag 2018-07-30 11:22:31 -03:00
Victor Shyba 1937eb17c4 adds a test for stopping 2018-07-30 11:22:31 -03:00
Victor Shyba f291ea515a add test for invalid content 2018-07-30 11:22:31 -03:00
Victor Shyba 56522f39c0 adds a CHANGELOG 2018-07-30 11:22:31 -03:00
Victor Shyba ff4aba9423 refactor mirroring 2018-07-30 11:22:31 -03:00
hackrush 9ab256df30 fixed improper error handling when data is not valid JSON (#1326) 2018-07-30 09:38:36 -04:00
Lex Berezhny 88b8ca8a49 Revert "negate wallet lock check in components"
This reverts commit 32507d7
2018-07-25 14:59:10 -04:00
Jack Robison 09f637496c Bump version 0.21.0rc2 --> 0.21.0rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-25 12:25:24 -04:00
Jack Robison 826d269e0e Merge branch 'blobs_mirroring' 2018-07-25 12:24:37 -04:00
Victor Shyba 5163baf9c1 improve changelog on changes section 2018-07-25 13:09:25 -03:00
Victor Shyba ab27203100 improve exception logging and add a docstring on the interaction between downloaders 2018-07-25 13:01:13 -03:00
Victor Shyba 88c2051605 set download_mirror conf from components change 2018-07-25 12:23:15 -03:00
Victor Shyba f510c2a433 improve logging from review 2018-07-25 12:15:52 -03:00
Victor Shyba 66982c86a9 adds changelog 2018-07-25 12:15:52 -03:00
Victor Shyba ec140d5d8a changes from code review 2018-07-25 12:15:52 -03:00
Victor Shyba ad96b006f9 adds http download support for blobs mirroring 2018-07-25 12:15:52 -03:00
Lex Berezhny 32507d74f7 negate wallet lock check in components 2018-07-25 01:28:19 -04:00
Jack Robison 2a00d00e44 docs 2018-07-24 21:56:15 -04:00
Jack Robison 40c5f6e3ab Bump version 0.21.0rc1 --> 0.21.0rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-24 21:41:58 -04:00
Jack Robison ae67918c8a Merge branch 'better-peer-ping' 2018-07-24 21:34:15 -04:00
Jack Robison 96e73f3476 changelog 2018-07-24 21:31:51 -04:00
Jack Robison c1b05402ef add address and port arguments to peer_ping
allows directly pinging the peer without first doing an iterative find
2018-07-24 21:30:54 -04:00
Jack Robison 1dd1bad9c2 Bump version 0.20.4 --> 0.21.0rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-24 21:21:52 -04:00
Jack Robison 9b873d2f48 Merge branch 'daemon-components' 2018-07-24 21:17:51 -04:00
Jack Robison 5984ae7ce9 fix 2018-07-24 21:10:53 -04:00
hackrush 903cd86cdd changelog 2018-07-24 20:02:08 -04:00
Jack Robison 5d9a187659 refactor required conditions and @requires decorator 2018-07-24 20:01:38 -04:00
Jack Robison a9c94ca22d move setup and _shutdown to AuthJSONRPCServer 2018-07-24 20:01:38 -04:00
Jack Robison a285db1b08 pylint 2018-07-24 20:01:38 -04:00
Jack Robison 13bf8125e3 status doc, better determination of is_first_run
-add skipped_components to status response
-re-add run_reflector_server to settings
2018-07-24 20:01:38 -04:00
Jack Robison b06dcf0a0d cancel starting components if the reactor is stopped before startup has finished
-don't block starting the dht component on having found enough peers, only on setting up the protocol
2018-07-24 20:01:38 -04:00
Jack Robison a800f6ddf0 update status command 2018-07-24 20:01:38 -04:00
Jack Robison a89306b6bf reorganize daemon startup
-fix loggly not using the share usage setting

-delete more
2018-07-24 20:01:38 -04:00
Jack Robison c3120e93cf delete unneeded daemon attributes 2018-07-24 20:01:38 -04:00
Jack Robison defe9506bb set daemon attribute for each component as they start 2018-07-24 20:01:38 -04:00
Jack Robison 55d3bb0ec3 remove auto_renew 2018-07-24 20:01:38 -04:00
hackrush 005a8b3008 allow utf-8 characters in Daemon.py
-amit really wants the shrug emoji ¯\_(ツ)_/¯
2018-07-24 20:01:38 -04:00
hackrush a741fdfc44 update daemon unit tests 2018-07-24 20:01:38 -04:00
hackrush 944b94aae8 update logging tests 2018-07-24 20:01:38 -04:00
hackrush 37eaf634c4 update functional tests 2018-07-24 20:01:38 -04:00
hackrush edcb06a415 update mocks, add test_Component_Manager 2018-07-24 20:01:38 -04:00
hackrush 62b50dc0ae move custom logger to lbrynet directory
-import on module level __init__
2018-07-24 20:01:38 -04:00
hackrush e7c57dcabc add components_to_skip setting 2018-07-24 20:01:38 -04:00
hackrush 094d9c6497 update lbrynet-cli 2018-07-24 20:01:38 -04:00
hackrush 3dc5a9de7b whitespace 2018-07-24 20:01:38 -04:00
hackrush 4e29041291 update status command 2018-07-24 20:01:38 -04:00
hackrush 6b6a29fdb7 update settings_set 2018-07-24 20:01:38 -04:00
hackrush 2d61ba629f add requires decorator 2018-07-24 20:01:38 -04:00
hackrush 5a2075019c update Wallet.check_locked 2018-07-24 20:01:38 -04:00
hackrush 944200ca8c add all the daemon components 2018-07-24 20:01:38 -04:00
hackrush 40d8e96811 remove functions and attributes in Session and Daemon that are now part of components
-rename attributes in daemon to use components
2018-07-24 20:01:38 -04:00
Jack Robison 7e8ca842a2 change ComponentManager to use instance methods rather than class methods
-add get_component method to ComponentManager

-add override_components kwargs to ComponentManager

-add skip_components to ComponentManager

-change component_manager attribute to exist on the AuthJSONRPCServer instance instead of the class
2018-07-24 20:01:38 -04:00
Jack Robison b808d08eb3 support callbacks to component setups 2018-07-24 20:01:38 -04:00
Jack Robison eb11da9b19 convert directory and SQLiteStorage setup to be a Component 2018-07-24 20:01:38 -04:00
Jack Robison 68b31a09b4 add daemon Component and ComponentManager classes 2018-07-24 20:01:38 -04:00
Lex Berezhny cab8416596 api doc generator (#1290)
script to generate docs/api.json
2018-07-24 19:36:09 -04:00
Electron - Mark Firth 94f4e6714a A More Informative Error Message (#1309)
Additional information added to the balance error message when editing a claim.
2018-07-20 15:41:24 -04:00
Jack Robison 2ddd6b051d Bump version 0.20.4rc4 --> 0.20.4
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-18 15:36:21 -04:00
Jack Robison c506fc5c75 lower peer finder timeout message to debug from warning 2018-07-18 15:32:45 -04:00
Jack Robison d306ece0d4 Bump version 0.20.4rc3 --> 0.20.4rc4
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-17 17:38:27 -04:00
Jack Robison d29f103910 pylint 2018-07-17 17:38:19 -04:00
Jack Robison 2386a3a79b Bump version 0.20.4rc2 --> 0.20.4rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-17 17:14:07 -04:00
Jack Robison ec51333b56 allow node udp interface to be configured 2018-07-17 17:13:33 -04:00
Jack Robison 75829812f3 Merge pull request #1305 from lbryio/fix_score_call
fix where it called the wrong score method
2018-07-17 17:02:35 -04:00
Victor Shyba 4d720370b1 fix where it called the wrong score method 2018-07-17 17:48:25 -03:00
Jack Robison 7d7dd665aa Bump version 0.20.4rc1 --> 0.20.4rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-17 13:31:37 -04:00
Jack Robison b6bee47481 changelog 2018-07-17 13:31:20 -04:00
Jack Robison 0ec8739d5a Merge branch 'fix_on_high_cpu_usage_on_incomplete_streams' 2018-07-17 13:24:06 -04:00
Victor Shyba 7d09d1402f changelog 2018-07-17 13:22:08 -04:00
Victor Shyba 5492ab9081 stop requesting peers which doesnt have a blob 2018-07-17 13:22:08 -04:00
Jack Robison 7f6bbc0648 Merge branch 'dht_shortlist_without_ignored' 2018-07-17 13:01:17 -04:00
Victor Shyba b1d4072c0b changelog 2018-07-17 12:57:57 -04:00
Victor Shyba 562d23df11 remove nodes that went ignored during iteration 2018-07-16 17:22:46 -03:00
Jack Robison 2c13c04f86 Bump version 0.20.3 --> 0.20.4rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-13 12:02:50 -04:00
Jack Robison 7365bfad9f Merge branch 'dht_failure_pruning' 2018-07-13 12:02:30 -04:00
Victor Shyba 1af1cf212c changelog 2018-07-13 11:39:44 -04:00
Victor Shyba 5c7d279104 prune failures during is_ignored calls 2018-07-12 18:24:59 -03:00
Victor Shyba cfe8e17223 filter out the bad node in results instead of punishing the node we trusted to contact 2018-07-12 18:24:18 -03:00
Thomas Zarebczan 91d4bf3cd9 Update ISSUE_TEMPLATE.md 2018-07-12 16:15:19 -04:00
Lex Berezhny df9642c325 removed slack notification from .travis.yml 2018-07-10 00:34:59 -04:00
Victor Shyba af98b0eb4d adds a changelog 2018-07-05 22:03:52 -03:00
Victor Shyba c589e89bf0 greedy search with exclude filtering 2018-07-05 22:03:52 -03:00
Lex Berezhny 284264962d reflector test fix 2018-07-05 19:51:15 -04:00
Lex Berezhny 6952c2c07e Revert "refactor lbrynet-daemon into modular components" (#1286)
* Revert "fix reflector test"

This reverts commit 6a15b51ac3.

* Revert "refactor lbrynet-daemon into modular components (#1164)"

This reverts commit 75a6ff269e.
2018-07-05 19:05:48 -04:00
Jack Robison 6a15b51ac3 fix reflector test 2018-07-05 15:40:13 -04:00
Jack Robison 75a6ff269e refactor lbrynet-daemon into modular components (#1164)
* add daemon Component and ComponentManager classes

* convert directory and SQLiteStorage setup to be a Component

* support callbacks to component setups

* Fixed typo in ComponentManager

* convert wallet to be Component

* Use storage from session.

* Remove create_session internal function and PEP8

* Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py

* Making DHT component(broken)

* Refactored classes to reduce redundancy in getting config setting

* DHT is now it's own component

* Fixed `test_streamify` test

* Fixed regression caused by removing `peer_manager` from session

* refactor ComponentManager and Component to use instance instead of class methods

* Hash announcer, file manager, stream identifier components

* Query Handler and server components

* Reflector Component

* Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯)

* All tests now passing

* Pylint fixes

* Oops(That's all you're gonna get :-P)

* Making decorators(WIP, commit so that I don't lose work)

* Decorator made and decorating of functions done(some other changes)

* import fixes and removed temporary test function

* Fixed new broken tests from daemon refactor

* Sanitization of modules

* Reworded errors

* wallet unlock condition checks, fixed breaking changes

* Rebased on amster and other crazy stuff

* Started writing tests

* Tests for component manager

* Fix Daemon Tests

* Fixed passing mutable args in init

* Using constants instead of strings. Added CHANGELOG.md

* Now components can be skipped by setting relevant config in file.

* P-Y-L-I-N-T #angry_emoji
2018-07-05 15:21:52 -04:00
ocornoc 148cc96025 Changed "cliam" to "claim" and added accurate info for nsfw. (#1285)
* Changed "cliam" to "claim" and added accurate info for nsfw.
* Update CHANGELOG.md
2018-07-05 12:00:43 -04:00
hackrush 14efa94382 Started to standardize the README.md file.
Closes #1252
2018-07-05 11:09:57 -04:00
Jack Robison da7e7503b9 Bump version 0.20.3rc2 --> 0.20.3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-03 14:32:38 -04:00
Jack Robison 67bedfafcb changelog 2018-07-03 14:29:46 -04:00
Jack Robison 5adcc86f40 Bump version 0.20.3rc1 --> 0.20.3rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-03 13:10:35 -04:00
Jack Robison a9971677f8 Merge branch 'stop_conditions_find_value' 2018-07-03 13:09:58 -04:00
Victor Shyba 8b6902fb00 adds a changelog 2018-07-03 13:49:35 -03:00
Victor Shyba 1cc7ce69ab set the peer search timeout as configured and raise it to 30s 2018-07-03 13:47:00 -03:00
Victor Shyba 7e0784d8e0 log a warning when peer search times out for real 2018-07-03 12:20:33 -03:00
Victor Shyba 023846703b adds a changelog 2018-07-03 11:53:21 -03:00
Victor Shyba 56613a71d5 let find value run until it finds a value 2018-07-03 11:51:39 -03:00
Jack Robison 2b0d4c1603 Bump version 0.20.2 --> 0.20.3rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-07-02 17:24:30 -04:00
Jack Robison a3e17f7916 Merge branch 'fix-find-close-nodes' 2018-07-02 17:24:09 -04:00
Jack Robison 929a0cc5af exclude self and querying node from closest contacts 2018-07-02 14:54:03 -04:00
Jack Robison b6289d101d fix improper sorting when getting the closest peers to a hash 2018-07-02 14:54:03 -04:00
Jack Robison 396542dc26 Merge branch 'self-store' 2018-07-02 14:52:38 -04:00
Jack Robison 7fe92d2df0 load all finished blobs into the node datastore 2018-07-02 14:31:09 -04:00
Jack Robison 03769b94b8 accept invalid store tokens for the first expire-time after startup
this is to better handle nodes storing to us after we've restarted our node
2018-07-02 13:26:38 -04:00
Jack Robison 07292363f9 Merge branch 'faster-channel-id-file-startup' 2018-07-02 10:49:13 -04:00
Jack Robison c2dc872a54 fix file manager startup locking up when there are many files in a channel 2018-06-29 21:45:30 -04:00
Jack Robison 8a0d0b44dd Merge branch 'fix-blob-list-keyerror' 2018-06-29 11:49:06 -04:00
Lex Berezhny accf161355 Merge pull request #1274 from lbryio/req-update
Removed jsonrpclib from requirements
2018-06-26 10:06:47 -05:00
hackrush a5411c7b29 Removed jsonrpclib from requirements 2018-06-26 10:31:04 -04:00
Jack Robison e0ceaba655 fix blob_list raising an error when blobs in a stream haven't yet been created 2018-06-25 16:47:23 -04:00
Jack Robison a1f3fc9f3c Merge pull request #1269 from lbryio/fix_stop_nonetype
Fix stop raising `NoneType' object has no attribute 'finished_deferred'
2018-06-25 16:42:07 -04:00
Victor Shyba 238a07e8e2 adds a changelog 2018-06-23 18:26:45 -03:00
Victor Shyba c5f4f58221 collect finished deferred before cancel call 2018-06-23 18:23:25 -03:00
Jack Robison 12d1b93b5d Bump version 0.20.2rc3 --> 0.20.2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-23 11:53:23 -04:00
Jack Robison 34febf75f9 Bump version 0.20.2rc2 --> 0.20.2rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-22 19:15:46 -04:00
Jack Robison 140c45b01f Revert "add download progress and is downloading flag to daemon status (#1266)"
This reverts commit 2fa2269
2018-06-22 18:54:22 -04:00
Akinwale Ariwodola 2fa2269cc7 add download progress and is downloading flag to daemon status (#1266) 2018-06-21 23:49:22 +01:00
Jack Robison 4bcf0ab7fb Bump version 0.20.2rc1 --> 0.20.2rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-20 17:07:54 -04:00
Jack Robison 3d5bef1595 Merge pull request #1264 from lbryio/faster-claim-sqlite
batch insert/update claims to sqlite
2018-06-20 17:06:29 -04:00
Jack Robison 61a7fd66bf batch insert/update claims to sqlite 2018-06-20 16:08:12 -04:00
Jack Robison 889b2facce Bump version 0.20.1 --> 0.20.2rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-20 16:01:24 -04:00
Jack Robison c38928ff58 docs 2018-06-19 14:09:21 -04:00
Jack Robison e435a9f722 Bump version 0.20.1rc5 --> 0.20.1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-19 14:05:16 -04:00
Jack Robison 10897ef33c clean up changelog 2018-06-19 14:02:24 -04:00
Jack Robison ba52574091 Bump version 0.20.1rc4 --> 0.20.1rc5
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-19 13:51:12 -04:00
Jack Robison cefe9db04d Merge pull request #1262 from lbryio/fix-1172
fix download being stopped if the sd blob downloaded and data did not start within the timeout
2018-06-19 13:50:16 -04:00
Jack Robison eb660e23b7 fix download being stopped if the sd blob downloaded and data did not start within the timeout
fixes https://github.com/lbryio/lbry/issues/1172
2018-06-19 13:38:25 -04:00
Jack Robison 64a961c88c Merge branch 'fix-1245' 2018-06-18 21:47:38 -04:00
Jack Robison 89bc26191c fix wallet startup locking up if a lbryumx server is unavailable 2018-06-18 18:42:48 -04:00
Jack Robison 75d260e8e6 fix appveyor 2018-06-18 16:53:02 -04:00
Jack Robison 01a597d5cd Merge pull request #1258 from lbryio/fix-950
fix blob client protocol not tearing itself down properly after a failure
2018-06-18 16:50:30 -04:00
Jack Robison 1615ca0ebd fix appveyor 2018-06-18 16:47:12 -04:00
Jack Robison 5edd8deb63 fix blob client protocol not tearing itself down properly after a failure 2018-06-18 15:30:35 -04:00
Lex Berezhny fa0dc128e6 Merge pull request #1257 from lbryio/fix-lex-floats
Fix approximations in bid during publish
2018-06-18 10:31:33 -05:00
hackrush 371b27b116 Fix approximations in bid during publish 2018-06-18 10:40:33 -04:00
Jack Robison db15169ca6 Bump version 0.20.1rc3 --> 0.20.1rc4
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-15 14:38:23 -04:00
Jack Robison 71e2c545fc Merge branch 'fix-1165' 2018-06-15 14:38:01 -04:00
Jack Robison 3270693a8f Bump version 0.20.1rc2 --> 0.20.1rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-15 14:36:31 -04:00
Jack Robison c26816c92d fix database error when updating a claim that we don't have a file for (https://github.com/lbryio/lbry/issues/1165) 2018-06-15 10:21:37 -04:00
Jack Robison e83ff47722 Bump version 0.20.1rc1 --> 0.20.1rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-15 10:19:43 -04:00
Jack Robison 4c97d2af2e fix request_needed_blobs for partially reflected streams 2018-06-15 10:10:49 -04:00
Jack Robison 42daf0fc3f Bump version 0.20.0 --> 0.20.1rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-14 17:55:10 -04:00
Jack Robison 408e1bff72 Merge branch 'fix-blob-race-condition' 2018-06-14 17:54:36 -04:00
Jack Robison 70355145a0 fix race condition when inserting a blob to the database 2018-06-14 15:15:16 -04:00
Jack Robison e6df378e9e fix token validation error when the dht node has just been started (https://github.com/lbryio/lbry/issues/1248) 2018-06-14 15:07:10 -04:00
Jack Robison e7deecc7ce Merge branch 'float_to_decimal' 2018-06-13 17:15:12 -04:00
Lex Berezhny dfd8e7212f updated changelog 2018-06-13 16:55:38 -04:00
Lex Berezhny 37bccdad9f added dependency: six 2018-06-13 16:52:35 -04:00
Lex Berezhny e11a2e6f85 publish accepts decimal string for the bid value 2018-06-13 16:52:35 -04:00
Jack Robison 21148995bd add tl;dr 2018-06-13 15:38:19 -04:00
Jack Robison 7030547849 Bump version 0.20.0rc14 --> 0.20.0
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-13 15:36:49 -04:00
Jack Robison 07190c5d69 clean up changelog 2018-06-13 15:10:09 -04:00
Lex Berezhny 04b03ef3fa added pull request links to CHANGELOG.md 2018-06-13 14:30:26 -04:00
Jack Robison df7a59f40a skip test on appveyor 2018-06-13 14:04:14 -04:00
Lex Berezhny 45bf5beb34 Merge pull request #1209 from jleute/master
fixes bug when saving server list to conf file (issue #1109)
2018-06-12 16:24:44 -05:00
jleute 0dbd15df7b Merge remote-tracking branch 'upstream/master' 2018-06-12 00:27:08 +02:00
jleute 905b07d714 Merge branch 'master' of https://github.com/jleute/lbry 2018-06-12 00:14:36 +02:00
jleute 9735373eef remove dependency on server_list_reverse 2018-06-12 00:10:23 +02:00
jleute 4b9079bf25 doesn't belong here. 2018-06-12 00:07:08 +02:00
jleute acbe3b0f65 updated test to compare original and saved config file 2018-06-12 00:06:49 +02:00
jleute ea7ef54174 remove unused import 2018-06-12 00:06:24 +02:00
jleute cfe5eb5d56 clean up temporary conf file 2018-06-12 00:06:21 +02:00
jleute e00238427b add assert to test_load_save_load_config_file test 2018-06-12 00:05:50 +02:00
jleute 5926b17871 Updated unittest for saving of server lists in the conf file 2018-06-12 00:05:49 +02:00
jleute 2e3ceedc17 added unit test for reversal of conversion of config file entries 2018-06-12 00:04:35 +02:00
jleute c411d8700b fixes #1109 2018-06-12 00:04:35 +02:00
Jack Robison 0ae89f01e6 Revert "WIP: feature/1098/sorted-claim-results" 2018-06-12 00:04:32 +02:00
Sergey Rozhnov d66bce814c refactored unit test for sort_claim_results 2018-06-12 00:03:30 +02:00
Sergey Rozhnov c290fb5908 refactored sorting of claims and unit tests 2018-06-12 00:02:48 +02:00
Sergey Rozhnov 156b132070 implemented additional test cases for claims sorting; updated changelog 2018-06-12 00:01:36 +02:00
Sergey Rozhnov b770b2de6a simplified claims comparison logic; refactored unit tests 2018-06-12 00:01:36 +02:00
Sergey Rozhnov c7ba82dc07 claim_list and claim_list_mine in Daemon return sorted results 2018-06-12 00:01:32 +02:00
Jack Robison c0c12adefa Bump version 0.20.0rc13 --> 0.20.0rc14
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-09 11:26:41 -04:00
Jack Robison 9219e5976e update default lbryum servers 2018-06-09 11:26:26 -04:00
jleute 0936bdbdd3 doesn't belong here. 2018-06-09 00:47:34 +02:00
jleute e381e4c063 Merge branch 'master' of https://github.com/lbryio/lbry 2018-06-09 00:41:38 +02:00
jleute c1fae45de6 Merge branch 'master' of https://github.com/jleute/lbry 2018-06-09 00:26:11 +02:00
Jack Robison 2619c396d9 Bump version 0.20.0rc12 --> 0.20.0rc13
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-08 13:50:35 -04:00
Jack Robison a1218ea565 Merge remote-tracking branch 'origin/fix_prm' 2018-06-08 13:50:23 -04:00
Jack Robison b92800158e Bump version 0.20.0rc11 --> 0.20.0rc12
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-08 13:49:36 -04:00
Jack Robison 76984dd8bd Merge branch 'fix-migrate-5-to-6' 2018-06-08 13:48:20 -04:00
Jack Robison d87d8875f6 fix migration error caused by using a current create tables query mid migration 2018-06-08 13:28:49 -04:00
Victor Shyba 8acf0be55b adds changelog 2018-06-08 11:59:23 -03:00
Victor Shyba 8598280654 fix base brm to prm typo 2018-06-08 11:16:21 -03:00
Jack Robison f0c732644a Bump version 0.20.0rc10 --> 0.20.0rc11
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-06-07 15:18:30 -04:00
Jack Robison d0fe4793fc Merge branch 'fix-dht-contact' 2018-06-07 15:16:16 -04:00
Jack Robison 665c73c38c changelog 2018-06-07 15:02:47 -04:00
Jack Robison 1d0106929c move daemon test to the same folder as the others 2018-06-07 15:02:47 -04:00
Jack Robison a821647fbc pylint and appveyor 2018-06-07 15:02:47 -04:00
Jack Robison 7f3ead67bf disable forced bucket refresh during join 2018-06-07 15:02:47 -04:00
Jack Robison f3e848b2e2 work around upnp bug
this fixes an incorrectly raised mapping conflict error which the router raises when there is a redirect for the same internal port and a different lan address.
2018-06-07 15:02:47 -04:00
Jack Robison b0e4fc4faa fix iterative find lockup 2018-06-07 15:02:47 -04:00
Jack Robison af096aee41 update test 2018-06-07 15:02:47 -04:00
Jack Robison ae631f05c3 fix teardown error 2018-06-07 15:02:47 -04:00
Jack Robison bc0da5e2d1 only use seeds in iterative bootstrap if no contacts are known yet 2018-06-07 15:02:47 -04:00
Jack Robison 16cb6d8657 remove Session._join_deferred 2018-06-07 15:02:47 -04:00
Jack Robison 44644673d7 add profiler 2018-06-07 15:02:47 -04:00
Jack Robison e8b402f998 remove deferredLock from iterativeFind
-fire the first iteration right away
2018-06-07 15:02:47 -04:00
Jack Robison adca5f5993 fix routing_table_get 2018-06-07 15:02:47 -04:00
Jack Robison 4fbaaac3f3 default new contacts to protocol version 0 2018-06-07 15:02:47 -04:00
Jack Robison db06191c33 reduce default concurrent announcers to 10
-lower rpc timeout to what it originally was
2018-06-07 15:02:47 -04:00
Jack Robison b0a741b1f4 fix hash announcer semaphore 2018-06-07 15:02:47 -04:00
Jack Robison 537df6c8ad log socket errors 2018-06-07 15:02:47 -04:00
Jack Robison 42eb172638 refactor announceHaveBlob
-add cached `token` to Contact objects to minimize findValue requests
-remove self_store, always store to remote contacts even if we're the closest known node to the hash
-move the store call and error handling from announceHaveBlob to a smaller function of its own
2018-06-07 15:02:47 -04:00
Jack Robison 0386bfadf0 update seed script 2018-06-07 15:02:47 -04:00
Jack Robison 9a63db4ec6 add protocol version to the dht and migrate old arg format for store 2018-06-07 15:02:47 -04:00
Jack Robison 7d21cc5822 pylint and more tests 2018-06-07 15:02:47 -04:00
Jack Robison cce3c8c7b5 increase kademlia rpc timeout to 8 seconds 2018-06-07 15:02:47 -04:00
Jack Robison 0e80123615 use 12 minutes instead of 15 as delay in contact_is_good 2018-06-07 15:02:47 -04:00
Jack Robison 30c4b160f1 use epoll reactor for seed node script 2018-06-07 15:02:47 -04:00
Jack Robison 659632b66c fix and update tests 2018-06-07 15:02:47 -04:00
Jack Robison 545930cea4 dht test environment fixes 2018-06-07 15:02:47 -04:00
Jack Robison 51b42da1c5 remove unused scripts 2018-06-07 15:02:47 -04:00
Jack Robison 9582b7fcf5 use maybe_ping for refreshing stale buckets and storing peers
-move store refresh to its own looping call
2018-06-07 15:02:47 -04:00
Jack Robison 945da5985e fix age used in datastore to determine if a value is expired 2018-06-07 15:02:47 -04:00
Jack Robison 921ee3c4c1 use refreshTimeout in getRefreshList 2018-06-07 15:02:47 -04:00
Jack Robison 8efede6ad6 maybe_ping bad and unknown contacts instead of only unknown 2018-06-07 15:02:47 -04:00
Jack Robison fb3aac15fa add optional delay argument to enqueue_maybe_ping, fix default value 2018-06-07 15:02:47 -04:00
Jack Robison 73e813f9ec verify key size in Distance 2018-06-07 15:02:47 -04:00
Jack Robison 0d23c68797 raise attribute error for non-rpc functions in Contact 2018-06-07 15:02:47 -04:00
Jack Robison ec1b6b2387 comments, cleaner key_bits constant 2018-06-07 15:02:47 -04:00
Jack Robison 29d5750371 pylint 2018-06-07 15:02:47 -04:00
Jack Robison 07f92014d7 omit bad contacts from getPeersForBlob
-refresh stored contacts to detect when they go offline or come back
2018-06-07 15:02:47 -04:00
Jack Robison 98e21cdba0 test re-join dht 2018-06-07 15:02:47 -04:00
Jack Robison c521120b17 update and fix hash announcer test 2018-06-07 15:02:47 -04:00
Jack Robison d02ed29e50 add kademlia store and expiration test 2018-06-07 15:02:47 -04:00
Jack Robison 470ebe2de3 remove unnecessary CallLaterManager from test_contact_rpc 2018-06-07 15:02:47 -04:00
Jack Robison b5f3ed5542 update contact expiration test and add re-join after expiration test 2018-06-07 15:02:47 -04:00
Jack Robison 877da78505 ping contacts right away during refresh instead of using PingQueue
-fixes contact expiration test
2018-06-07 15:02:47 -04:00
Jack Robison a952d2d2c8 reset _listeningPort and _listening Deferred on teardown 2018-06-07 15:02:47 -04:00
Jack Robison 3dfc6bd2cc update CallLaterManager to be an object 2018-06-07 15:02:47 -04:00
Jack Robison 760417ff3a pylint 2018-06-07 15:02:47 -04:00
Jack Robison 7da70cc02d test peer expiration 2018-06-07 15:02:47 -04:00
Jack Robison d3f4155a2b add unit tests for contact_is_good 2018-06-07 15:02:47 -04:00
Jack Robison d250e4d91a add iterative find test 2018-06-07 15:02:47 -04:00
Jack Robison 950ec5bc9a update mocks and dht tests
-reorganize dht tests
2018-06-07 15:02:47 -04:00
Jack Robison e1079a0c0f add seed node script and monitor tool, update dht monitor script 2018-06-07 15:02:47 -04:00
Jack Robison bdd6f948ca add port to routing_table_get 2018-06-07 15:02:47 -04:00
Jack Robison cc32d987b2 update peer_list 2018-06-07 15:02:47 -04:00
Jack Robison 4f72098cad use PingQueue to try refresh all contacts 2018-06-07 15:02:47 -04:00
Jack Robison 9920ff59d4 force KBucket refreshes 2018-06-07 15:02:47 -04:00
Jack Robison ae22468fec fix CallLaterManager trying to remove pending calls multiple times 2018-06-07 15:02:47 -04:00
Jack Robison aee7a3aa38 simplify announceHaveBlob, remove unused getPeersForBlob 2018-06-07 15:02:47 -04:00
Jack Robison 372fb45e06 refactor dht bootstrap
after finding the closest nodes try to populate the buckets out by looking up random ids in their key ranges
2018-06-07 15:02:47 -04:00
Jack Robison c65274e9e5 add PingQueue to KademliaProtocol 2018-06-07 15:02:47 -04:00
Jack Robison 1adf4f7818 fix constant used to check if a bucket is fresh 2018-06-07 15:02:47 -04:00
Jack Robison 05241012a2 update contact replacement in KBucket to follow BEP0005
http://www.bittorrent.org/beps/bep_0005.html
2018-06-07 15:02:47 -04:00
Jack Robison cf3359044d fix conditions for when a kbucket should be split
https://stackoverflow.com/a/32187456
2018-06-07 15:02:47 -04:00
Jack Robison 5631a24881 improve findCloseNodes, choose closest contacts from higher and lower buckets 2018-06-07 15:02:47 -04:00
Jack Robison c654bfe296 use reactor clock in TreeRoutingTable instead of time module 2018-06-07 15:02:47 -04:00
Jack Robison d4e28216a0 sort KBucket.getContacts 2018-06-07 15:02:47 -04:00
Jack Robison 95ed1e044b raise TransportNotConnected instead of logging a warning
-add a _listening Deferred to KademliaProtocol which is called when the protocol is started
2018-06-07 15:02:47 -04:00
Jack Robison f1e0a784d9 refactor iterativeFind, move to own file 2018-06-07 15:02:47 -04:00
Jack Robison e5703833cf prevent duplicate entries in the datastore 2018-06-07 15:02:47 -04:00
Jack Robison 23c202b5e4 refactor Contact class, DHT RPCs, and Contact addition/removal
-track contact failures, last replied, and last requested. use this to provide a 'contact_is_good' property on Contact objects
-ensure no duplicate contact objects are created
-remove confusing conflation of node id strings with Contact objects, update docstrings
-move RPC failure tracking to a callback/errback pair in sendRPC (so the contact is only updated once)
-handle seed nodes during the join sequence by setting their node ids after they initially reply to our ping
-name all of the kademlia RPC keyword args, remove confusing **kwargs and dictionary parsing
-add host ip/port to DHT send/receive logging to make the results comprehensible when running many nodes at once
2018-06-07 15:02:47 -04:00
Jack Robison ad2dcf0893 add the parent node id to KBucket 2018-06-07 15:02:47 -04:00
Jack Robison 406ddaa4ef use base class to simplify Node init
-add looping call helpers which use the same clock as the Node
2018-06-07 15:02:47 -04:00
Jack Robison e52689a33d remove OptimizedTreeRoutingTable for now, use TreeRoutingTable 2018-06-07 15:02:47 -04:00
Jack Robison d65dc0aec3 disable loading DictDataStore in Node.__init__
-to be re-done when the datastore uses sqlite
2018-06-07 15:02:47 -04:00
Jack Robison 159e153393 make DataStore clock mockable 2018-06-07 15:02:47 -04:00
Jack Robison 9ed08f8fc9 remove unused constant 2018-06-07 15:02:47 -04:00
Jack Robison a503a800ca disable Cryptonator exchange rate feed 2018-06-07 15:02:47 -04:00
Jack Robison b673c508cc disable NegotiatedPaymentRateManager, use OnlyFreePaymentsManager for now 2018-06-07 15:02:47 -04:00
Jack Robison 4bd9f3bd68 remove popular hash tracking, simplify DHTPeerFinder 2018-06-07 15:02:47 -04:00
Jack Robison e6ffd7caf8 remove hashwatcher 2018-06-07 15:02:47 -04:00
Jack Robison e4219b9cfc Merge pull request #1226 from shyba/refactor_sql_calls
refactor: simplify add_completed_blob + changelog
2018-05-31 13:40:33 -04:00
Victor Shyba b91181e640 refactor: simplify add_completed_blob + changelog 2018-05-30 16:19:49 -03:00
Lex Berezhny adac6b21e3 Merge pull request #1174 from kafene/feature/file_list_sorting
Implement sorting in file_list
2018-05-30 00:03:43 -04:00
Lex Berezhny 23df1d0316 Merge branch 'master' into feature/file_list_sorting 2018-05-29 22:47:09 -04:00
Lex Berezhny bd1460e608 Merge pull request #1213 from lbryio/twisted-ssl
added certifi package for Twisted SSL verification on Windows
2018-05-29 22:44:23 -04:00
Lex Berezhny 3d17c92bd3 moved import certifi to run only on windows 2018-05-29 22:36:37 -04:00
Akinwale Ariwodola 0d35621a9a updated changelog 2018-05-29 22:36:37 -04:00
Akinwale Ariwodola c4550ef5b6 remove unused import in lbrynet.daemon.Daemon 2018-05-29 22:36:37 -04:00
Akinwale Ariwodola 97547e904e add os and sys imports 2018-05-29 22:36:37 -04:00
Victor Shyba 03968f377e change fix position to before anything else starts 2018-05-29 22:36:37 -04:00
Akinwale Ariwodola 513f5d0568 added certifi package for Twisted SSL verification on Windows 2018-05-29 22:36:37 -04:00
Jack Robison 02bcc98a6a Bump version 0.20.0rc9 --> 0.20.0rc10
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-05-29 15:56:58 -04:00
Jack Robison e3265714ee Merge remote-tracking branch 'origin/1193' 2018-05-29 15:25:17 -04:00
Jack Robison 4b766eeea9 Merge remote-tracking branch 'origin/linux_analytics' 2018-05-29 15:20:46 -04:00
Lex Berezhny 6a344539a0 moved import distro into conditional statement that only runs on Linux 2018-05-29 14:49:06 -04:00
Jack Robison 9b1e687b9b pylint 2018-05-29 14:36:00 -04:00
Sergey Rozhnov 11b882879a predictable result sorting for claim_list and claim_list_mine 2018-05-29 13:07:11 -04:00
Lex Berezhny 09e272a642 style fix 2018-05-29 12:45:42 -04:00
Lex Berezhny d950a62200 include the new linux analytics in track events 2018-05-28 18:47:56 -04:00
Lex Berezhny 89729860cb added analytics change to CHANGELOG 2018-05-28 18:19:33 -04:00
Lex Berezhny 1b027fae87 add some extra linux meta data to analytics 2018-05-28 18:07:23 -04:00
Jack Robison 187f920961 disable Daemon headers check for now 2018-05-28 10:57:48 -04:00
Jack Robison 25c36346b6 Revert "WIP: feature/1098/sorted-claim-results" 2018-05-23 19:16:03 +02:00
Sergey Rozhnov e220a57f0e refactored unit test for sort_claim_results 2018-05-23 19:16:03 +02:00
Sergey Rozhnov 65fed3f5aa refactored sorting of claims and unit tests 2018-05-23 19:16:03 +02:00
Sergey Rozhnov eb164d9ac7 implemented additional test cases for claims sorting; updated changelog 2018-05-23 19:16:03 +02:00
Sergey Rozhnov 332b3e1fef simplified claims comparison logic; refactored unit tests 2018-05-23 19:16:03 +02:00
Sergey Rozhnov 4aa503b863 claim_list and claim_list_mine in Daemon return sorted results 2018-05-23 19:16:03 +02:00
Jack Robison bc6fd892a2 Merge remote-tracking branch 'origin/revert-1208-feature/1098/sorted-claim-results' 2018-05-22 17:19:32 -04:00
Jack Robison 6d4b678b96 Revert "WIP: feature/1098/sorted-claim-results" 2018-05-22 17:15:34 -04:00
jleute 092581bf32 updated test to compare original and saved config file 2018-05-21 18:50:25 +02:00
Jack Robison 13996da1a4 Merge pull request #1208 from tau3/feature/1098/sorted-claim-results
WIP: feature/1098/sorted-claim-results
2018-05-21 09:24:49 -04:00
kafene 76a7cc4e58 refactor file list sorting per request 2018-05-21 00:24:18 -04:00
kafene 1790393273 remove unused attrgetter import 2018-05-21 00:04:27 -04:00
kafene 5069351287 refactor file_list sort to allow sorting by any field 2018-05-21 00:04:27 -04:00
kafene 72bae90e06 use range instead of xrange, list is only 10 elements and range works in python3 2018-05-21 00:04:27 -04:00
kafene ee4b47aedc add directive to install packages in requirements_testing.txt for travis 2018-05-21 00:04:27 -04:00
kafene 852354719f add unit test for file list sorting 2018-05-21 00:04:27 -04:00
kafene eec3734d2f some semantic variable renaing 2018-05-21 00:04:27 -04:00
kafene 89996f929d add unit test package requirements - mock and Faker 2018-05-21 00:04:27 -04:00
kafene ce1e7d669e add changelog entry for file_list sort option 2018-05-21 00:04:27 -04:00
kafene e7b22521ee remove pprint import 2018-05-21 00:04:27 -04:00
kafene 326d0cb8a3 initial naive implementation of file_list sorting 2018-05-21 00:04:21 -04:00
jleute 93ee9fcff1 remove unused import 2018-05-19 09:44:15 +02:00
jleute 3add623ac9 clean up temporary conf file 2018-05-19 02:42:45 +02:00
jleute 500998b893 add assert to test_load_save_load_config_file test 2018-05-19 02:15:37 +02:00
jleute b543c04572 Updated unittest for saving of server lists in the conf file 2018-05-19 01:32:51 +02:00
Sergey Rozhnov 13353bcfe4 refactored unit test for sort_claim_results 2018-05-18 18:51:28 +04:00
Sergey Rozhnov 40bcf96c3d refactored sorting of claims and unit tests 2018-05-18 12:53:32 +04:00
jleute 348544428f added unit test for reversal of conversion of config file entries 2018-05-18 02:54:40 +02:00
jleute b16402e094 fixes #1109 2018-05-17 23:41:23 +02:00
Victor Shyba b48492c1d6 verify streams on a new migration instead 2018-05-17 12:29:12 -03:00
Sergey Rozhnov 3d0f74c8ce implemented additional test cases for claims sorting; updated changelog 2018-05-17 13:49:09 +04:00
Sergey Rozhnov a7a5bb8887 simplified claims comparison logic; refactored unit tests 2018-05-17 13:36:32 +04:00
Sergey Rozhnov 208284a0f8 claim_list and claim_list_mine in Daemon return sorted results 2018-05-16 18:29:44 +04:00
Thomas Zarebczan ff76cf5c3b Merge pull request #1201 from Fornost461/patch-1
grammar (missing word)
2018-05-14 17:29:49 -04:00
Fornost461 b2469801ab grammar (missing word) 2018-05-13 14:46:24 +02:00
Victor Shyba e1f4623a65 add changelog 2018-05-11 12:17:55 -03:00
Victor Shyba b6cedfec56 batch-start the file manager 2018-05-11 12:17:55 -03:00
Victor Shyba ca86af736e add batching support to get_supports and tests 2018-05-11 12:17:55 -03:00
Victor Shyba d55ded78ee get claim with channel_name from a stream hash in a single query 2018-05-11 12:17:55 -03:00
Victor Shyba acd330aa2a get claim info and channel name using a single query 2018-05-11 12:17:55 -03:00
Victor Shyba df735252e5 verify streams only after migration 2018-05-11 12:17:55 -03:00
Jack Robison b3bf193188 Merge remote-tracking branch 'origin/treq_and_cryptography' 2018-05-11 09:22:28 -04:00
Victor Shyba a0a7187f7d add integrity checks after s3 download as well 2018-05-08 16:09:06 -03:00
Jack Robison 43e4ad66d2 Merge branch 'improve-re-reflect' 2018-05-08 14:44:54 -04:00
Jack Robison aa4240de85 track successful reflector uploads in sqlite to minimize how many streams are attempted by auto re-reflect
-increase the default `auto_re_reflect_interval` to a day
2018-05-08 14:41:56 -04:00
Thomas Zarebczan 3a9f9a464e update changelog with 19.3 2018-05-08 11:32:05 -04:00
Victor Shyba d2e7ac5b6e update changelog for the integrity check and partial s3 downloads 2018-05-07 15:10:27 -03:00
Victor Shyba 2299098884 add integrity check for the headers file 2018-05-07 15:10:19 -03:00
Victor Shyba fd04c607b2 typos and fixes from code review 2018-05-07 13:11:31 -03:00
Victor Shyba 805bfbd18e remove point trader testing client 2018-05-07 11:49:54 -03:00
Victor Shyba 3ee0f9756d add treq as a dependency + pep8 2018-05-05 20:58:10 -03:00
Victor Shyba c90140f701 make the exchange manager use treq instead of requests 2018-05-05 02:33:37 -03:00
Victor Shyba 8bfc35753a update changelog 2018-05-05 02:20:55 -03:00
Victor Shyba e170f3db3e remove txrequests dependency 2018-05-05 02:20:21 -03:00
Victor Shyba 7f88dda0ae use treq on loggly 2018-05-05 02:18:03 -03:00
Victor Shyba 0bf65836c7 resume download from S3 instead of starting from scratch 2018-05-05 02:05:02 -03:00
Victor Shyba 3982e15091 download headers from s3 using treq 2018-05-05 01:16:26 -03:00
Victor Shyba d03fc80eac make analytics use treq 2018-05-05 00:50:42 -03:00
Victor Shyba 1077d8f937 update CHANGELOG.md 2018-05-05 00:09:24 -03:00
Victor Shyba 820789936f bump cryptography==2.2.2 2018-05-05 00:08:02 -03:00
Victor Shyba 44b08ae6c9 remove pycrypto 2018-05-04 23:59:08 -03:00
Victor Shyba fab932abb6 replace pycrypto->cryptography 2018-05-04 23:56:51 -03:00
Victor Shyba b389e59452 replace pycrypto->cryptography on tests 2018-05-04 22:12:43 -03:00
Jack Robison cc34c4b95f Merge pull request #1189 from lbryio/faster-headers
magic
2018-05-04 11:10:43 -04:00
Thomas Zarebczan df2339231a magic 2018-05-04 00:47:08 -04:00
Jack Robison 6977d6944e show local, remote, and s3 header heights in the log 2018-05-02 17:25:00 -04:00
Jack Robison f1e3682302 socket might not have TCP_KEEPx attributes 2018-05-02 17:16:00 -04:00
Jack Robison 28386e8935 Merge branch 'faster-headers' 2018-05-02 15:08:38 -04:00
Jack Robison f42733ecba fix tests 2018-05-02 15:00:57 -04:00
Jack Robison 129d2687b9 download headers from s3 when more than 10 chunks behind 2018-05-02 15:00:50 -04:00
Alex Grintsvayg f32861923e update blob download script 2018-05-02 13:23:57 -04:00
Jack Robison 1f1b348638 fix handling ancient non-decodable claims 2018-04-19 17:37:23 -04:00
Jack Robison 8b7e86e046 fix appveyor? 2018-04-19 17:36:26 -04:00
Jack Robison fe6a850fc5 appveyor 2018-04-19 17:11:27 -04:00
Jack Robison 9c25a37d4d Bump version 0.20.0rc8 --> 0.20.0rc9
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-04-19 17:04:48 -04:00
Jack Robison 1e87767a0c fix appveyor 2018-04-19 16:56:48 -04:00
Jack Robison 487ef6516e fix reflector test 2018-04-19 16:35:15 -04:00
Jack Robison 4a1f3a79a5 fix mock announcer 2018-04-19 16:18:19 -04:00
Jack Robison b4b1f1a2c1 Merge branch 'fix-announce-lockup' 2018-04-19 16:05:56 -04:00
Jack Robison 11aad04c31 datagram decode error logging 2018-04-19 15:47:42 -04:00
Jack Robison b89823c968 fix blob announce locking up the daemon process 2018-04-19 15:47:19 -04:00
Jack Robison baf55165aa lint 2018-04-19 14:35:11 -04:00
Jack Robison 769bc2e36e Revert "elif"
This reverts commit c3e047a2aa.
2018-04-19 14:34:49 -04:00
Alex Grintsvayg c3e047a2aa elif 2018-04-19 13:47:24 -04:00
Jack Robison 1c793931ed Bump version 0.20.0rc7 --> 0.20.0rc8
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-04-19 12:31:54 -04:00
Jack Robison fa49b40389 fix claim_show 2018-04-19 12:31:36 -04:00
Jack Robison 89b4487eb9 Bump version 0.20.0rc6 --> 0.20.0rc7
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-04-19 11:05:42 -04:00
Jack Robison 450bed6350 fix kwargs bug 2018-04-19 11:00:44 -04:00
Alex Grintsvayg f8e418fb44 undeprecated report_bug while app team works on a new bug reporting flow 2018-04-18 14:58:58 -04:00
hackrush 8aa0de0214 Merge pull request #1167 from lbryio/travis-fix
Fix travis going bonkers over pip errors
2018-04-18 10:19:02 -04:00
hackrush f3f6100e43 Fix travis going bonkers over pip errors 2018-04-18 19:43:03 +05:30
Alex Grintsvayg d973d1270d dht script 2 2018-04-17 13:47:56 -04:00
Alex Grintsvayg d8b424e694 docs 2018-04-17 13:47:09 -04:00
Alex Grintsvayg adcd6edfcb a very simple dht node script 2018-04-17 13:40:42 -04:00
Jack Robison 2655797692 move docs tests to own file 2018-04-12 14:04:22 -04:00
Niko c089f1954a fix documentation for blob_reflect 2018-04-12 13:27:06 -04:00
Niko 0a14d51b96 fix documentation for peer_ping command 2018-04-12 13:08:58 -04:00
Jack Robison bb7e17e54a Bump version 0.20.0rc5 --> 0.20.0rc6
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-04-03 13:21:17 -04:00
Jack Robison ad6a2ccc1e Merge branch 'daemon-dht-commands' 2018-04-03 13:20:52 -04:00
Jack Robison 5cea031f38 return list of dictionaries from peer_list, include peer node ids 2018-04-03 13:10:30 -04:00
Jack Robison 2903ccaeb4 add peer_ping 2018-04-03 13:10:30 -04:00
Jack Robison eae97a8133 off by one 2018-04-03 13:08:36 -04:00
Jack Robison bc3128ae3b Bump version 0.20.0rc4 --> 0.20.0rc5
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-04-03 12:34:45 -04:00
Jack Robison 4e7bdafdcf fix positional and keyword args 2018-04-03 12:27:49 -04:00
Jack Robison 12d378af13 show not connected error in lbrynet-cli when using auth 2018-04-03 12:25:18 -04:00
Jack Robison ca73d632d3 Bump version 0.20.0rc3 --> 0.20.0rc4
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-04-02 10:13:02 -04:00
Jack Robison e673dc6cfd Merge branch 'auth-requests' 2018-04-02 10:12:28 -04:00
Jack Robison 743ae59d54 fix lbrynet-cli when using authentication
-add explanation of daemon authentication to AuthJSONRPCServer docstring
-remove auth_required decorator, use auth for all api methods if use_authentication is true
-fix issues with the command line --http-auth flag to lbrynet-daemon and the use_http_auth setting in the config file
2018-04-01 18:31:03 -04:00
Jack Robison ea5190aa9a Bump version 0.20.0rc2 --> 0.20.0rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-29 15:06:41 -04:00
Jack Robison 51662b1e5b re-add hash_queue_size function 2018-03-29 15:06:08 -04:00
Jack Robison 789ca3693f Bump version 0.20.0rc1 --> 0.20.0rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-29 12:20:54 -04:00
Jack Robison 8c2b626f97 fix cli kwargs 2018-03-29 12:19:41 -04:00
Jack Robison 0279d37b32 fix status docstring 2018-03-29 12:19:18 -04:00
Jack Robison f4aca40ff9 Bump version 0.19.3rc2 --> 0.20.0rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-29 09:55:34 -04:00
Jack Robison 08e9f9ac0e Bump version 0.19.3rc1 --> 0.19.3rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-29 09:50:14 -04:00
Jack Robison 9a9b2f47ce pass sd_hash to reflector client factory instead of looking it up 2018-03-29 09:46:34 -04:00
Jack Robison b9600101ae Bump version 0.19.2 --> 0.19.3rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-28 19:36:39 -04:00
Jack Robison 31e032bb55 Merge branch 'dht-inlinecallbacks-refactor' 2018-03-28 19:35:27 -04:00
Jack Robison 65b918644d changelog 2018-03-28 19:25:55 -04:00
Jack Robison 492858596e add single_announce column to blob table
-remove deprecated blob_announce_all function
-remove announce_all parameter to blob_announce
-change blob_announce to be asynchronous
2018-03-28 18:50:24 -04:00
Jack Robison dff1fd3fe9 logging, raise default concurrent announcers 2018-03-28 18:50:24 -04:00
Jack Robison eabf4a0e40 remove delay from udp write 2018-03-28 18:50:24 -04:00
Jack Robison 5bab6f7d39 remove bandwidth stats 2018-03-28 18:50:24 -04:00
Jack Robison a41bbd5e27 pylint and tests 2018-03-28 18:50:18 -04:00
Jack Robison 43d3f7c087 add concurrent_announcers to config 2018-03-28 15:53:51 -04:00
Jack Robison c5bf64cf0a refactor DHTHashAnnouncer
-remove hash_announcer from Node and DiskBlobManager
-remove announcement related functions from DiskBlobManager
-update SQLiteStorage to store announcement times and provide blob hashes needing to be announced
-use dataExpireTimeout from lbrynet.dht.constants for re-announce timing
-use DeferredSemaphore for concurrent blob announcement
2018-03-28 15:53:51 -04:00
Jack Robison ea0ea704a2 refactor iterativeAnnounceHaveBlob
-change to only self_store if the number of contacts to store to is less than k and we are the closest node to the hash
2018-03-28 15:53:51 -04:00
Jack Robison 14f9bb7b82 log EWOULDBLOCK 2018-03-28 15:53:51 -04:00
Jack Robison a8025b02c6 log invalid vs missing token 2018-03-28 15:53:51 -04:00
Jack Robison 333d70860b add last_announced_time to blob table 2018-03-28 15:53:50 -04:00
Lex Berezhny 267f50474b removing more references to PTCWallet 2018-03-28 15:53:50 -04:00
Lex Berezhny 866f220d9b removed PTCWallet 2018-03-28 15:53:50 -04:00
Jack Robison a96d827c0f use reactor time in Delay 2018-03-28 15:53:50 -04:00
Jack Robison 4eab77fa10 safe start_listening 2018-03-28 15:53:50 -04:00
Jack Robison 5013426e02 logging and docstring 2018-03-28 15:53:50 -04:00
Jack Robison cb09be5336 remove unused stuff 2018-03-28 15:53:50 -04:00
Lex Berezhny 339e666f38 + Wallet.wait_for_tx_in_wallet 2018-03-28 15:53:49 -04:00
Jack Robison ebe5dd0e68 better ping test 2018-03-28 15:53:49 -04:00
Jack Robison 1db44d5fb6 rename variable 2018-03-28 15:53:49 -04:00
Jack Robison fd7a771f66 add sanity check to CreateEncryptedFileTest 2018-03-28 15:53:49 -04:00
Jack Robison 5628d0825b add CallLaterManager 2018-03-28 15:53:49 -04:00
Jack Robison 88970cb0a8 move peer manager class to lbrynet.core 2018-03-28 15:53:49 -04:00
Jack Robison d2a6dd3ed3 add dht functional tests 2018-03-28 15:53:49 -04:00
Jack Robison 1eff35ce76 update dht unit tests to use task.Clock 2018-03-28 15:53:49 -04:00
Jack Robison 0ab5dd28bc update tests 2018-03-28 15:53:48 -04:00
Jack Robison 2e30ce9ae5 add mock udp transport layer 2018-03-28 15:53:48 -04:00
Jack Robison 87c69742cd log packet encoding errors and warn if the transport is not connected 2018-03-28 15:53:48 -04:00
Jack Robison b4bc5e2110 cancel callLater on error or timeout 2018-03-28 15:53:48 -04:00
Jack Robison df78f7ff9f add response assertion to announce_to_peer 2018-03-28 15:53:48 -04:00
Jack Robison bdba263224 catch TimeoutError in _IterativeFindHelper 2018-03-28 15:53:48 -04:00
Jack Robison 16fcc3f5c1 findValue inlinecallbacks refactor 2018-03-28 15:53:48 -04:00
Jack Robison 43896c8d17 refactor joinNetwork into smaller functions
-try to re-join network if no contacts are known
2018-03-28 15:53:47 -04:00
Jack Robison 04e76443c6 move dht node component setup from Session into Node 2018-03-28 15:53:47 -04:00
Jack Robison e6caedac91 remove DHTHashSupplier class, move former functions into DHTHashAnnouncer 2018-03-28 15:53:47 -04:00
Jack Robison 6666468640 add reactor arguments to Node
-adds reactor (clock) and reactor functions listenUDP, callLater, and resolve as arguments to Node.__init__
-set the reactor clock on LoopingCalls to make them easily testable
-convert callLater manage loops to LoopingCalls
2018-03-28 15:53:47 -04:00
Jack Robison efaa97216f move dht node setup back into node class 2018-03-28 15:53:47 -04:00
Jack Robison e30ea50ef4 more 2018-03-28 15:53:47 -04:00
Jack Robison 3296c0fb3d move dht related classes to lbrynet.dht 2018-03-28 15:53:47 -04:00
Jack Robison 54a152fa8e fix merge conflict in dht tests 2018-03-28 15:53:46 -04:00
Jack Robison e6cc9b0de1 changelog 2018-03-28 15:53:46 -04:00
Kay Kurokawa 9088d152b5 better to keey track of retry count in function instead of unbounded dictionary 2018-03-28 15:53:46 -04:00
Kay Kurokawa 4cb461601e result must be set here, otherwise it will not be defined when used later. Add test for it 2018-03-28 15:53:46 -04:00
Kay Kurokawa 0425c95b68 No need for clock now in test for DHTHashAnnouncer 2018-03-28 15:53:46 -04:00
Kay Kurokawa 75b977dff9 we just have one supplier not a list of suppliers 2018-03-28 15:53:46 -04:00
Kay Kurokawa 60c7261871 add single_hash_announce_duration as a field that gets return for API call status for dht status 2018-03-28 15:53:46 -04:00
Kay Kurokawa 0f3385e4dc make the single hash announce duration adjustable in DHTHashSupplier 2018-03-28 15:53:46 -04:00
Jack Robison ad6a2bef7f handle error from old clients with a broken ping command 2018-03-28 15:53:45 -04:00
Jack Robison 446c3a88dc refactor DHTHashAnnouncer and iterativeAnnounceHaveBlob
-use looping call for running manage function rather than a scheduled
callLater

-track announce speed

-retry store requests that failed up to 3 times

-return a dict of {blob_hash: [storing_node_id]} results from
_announce_hashes

_refreshRoutingTable inline cb refactor

-add and use DeferredLockContextManager

-don't trap errback from iterativeFindNode in iterativeAnnounceHaveBlob
2018-03-28 15:53:45 -04:00
Jack Robison 67ef8be7b7 convert node manage function to a looping call 2018-03-28 15:53:45 -04:00
Jack Robison ecbe4113ce move Distance to own file 2018-03-28 15:53:45 -04:00
Jack Robison 528f168dc1 docs 2018-03-28 15:40:01 -04:00
Jack Robison b75c8d96ce Bump version 0.19.2rc3 --> 0.19.2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-28 15:35:46 -04:00
Jack Robison cb83531d05 Bump version 0.19.2rc2 --> 0.19.2rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-28 15:15:47 -04:00
Jack Robison dc65fd148e Bump version 0.19.2rc1 --> 0.19.2rc2
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-26 13:36:51 -04:00
Jack Robison 439a0f8778 fix docstrings
-add docopt unit test
2018-03-26 13:20:33 -04:00
Jack Robison a9fd19e6cf Merge pull request #1157 from lbryio/fix-1156
Fixed regression in stream_availablity command
2018-03-26 10:20:08 -04:00
hackrush aa7b440989 Fixed regression in stream_availablity command 2018-03-26 19:43:06 +05:30
Jack Robison 1aeba976da Merge pull request #1139 from lbryio/no-short-flags
Remove flags decorator from server.py as short flags are no longer used
2018-03-23 15:05:25 -04:00
Jack Robison f0742675b8 Merge branch 'master' into no-short-flags 2018-03-23 14:59:53 -04:00
Alex Grintsvayg 9ee5a3511d missed this in dht-compat merge 2018-03-23 12:50:18 -04:00
Jack Robison 69657750b8 Bump version 0.19.1 --> 0.19.2rc1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-22 17:04:12 -04:00
Jack Robison cb069ca002 add blob_reflect 2018-03-22 16:54:29 -04:00
Jack Robison 3085e28490 don't try to resend failed blobs to reflector
-fixes infinite loop where client keeps trying to send failing blobs, which may be failing because they are invalid

-return list of reflected blob hashes from BlobReflectorClient
2018-03-22 16:53:16 -04:00
Jack Robison 679c2f403f fix incorrectly raised DownloadCanceledError for already verified blobs
-make InvalidDataError message clearer
-fix BlobReflectorClient result
2018-03-22 13:23:12 -04:00
Alex Grin 0cbe11b263 Update INSTALL.md 2018-03-21 08:25:24 -04:00
Alex Grintsvayg 539881b5ad oops 2018-03-20 11:38:02 -04:00
Thomas Zarebczan ba8248887a cleanup changelog 2018-03-20 11:29:30 -04:00
Jack Robison ef7a360ee6 docs 2018-03-20 10:44:17 -04:00
Jack Robison de280b5ebf Bump version 0.19.1rc7 --> 0.19.1
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-20 10:15:50 -04:00
Jack Robison f283a2116e changelog 2018-03-20 10:15:01 -04:00
Jack Robison 113cc8bb15 Bump version 0.19.1rc6 --> 0.19.1rc7
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-19 14:31:17 -04:00
Jack Robison 7096d54e60 Merge remote-tracking branch 'origin/fix-slow-file-manager-setup' 2018-03-19 14:28:57 -04:00
Jack Robison b28bdbd752 rename existing reupload.reflect_stream --> reupload.reflect_file, add a reupload.reflect_stream function 2018-03-19 13:55:11 -04:00
Jack Robison f94a9e8729 start lbry files in parallel 2018-03-19 13:46:27 -04:00
Jack Robison 43c73b9abf fix slow sqlite query 2018-03-19 13:31:05 -04:00
hackrush 9a3eec0950 Sanitize changelog 2018-03-18 16:13:45 +05:30
hackrush 3724fa1810 Removed last flag decorator 2018-03-18 16:11:19 +05:30
hackrush f7f6770120 Removed flags decorator as short flags are no longer used. 2018-03-18 16:11:19 +05:30
hackrush 1d9c996217 Merge pull request #1148 from lbryio/channel-bid
Update channel balance checking to match claim balance checking
2018-03-17 09:05:08 +05:30
hackrush 2b69d2fad0 Fix pylint error 2018-03-16 21:42:40 +05:30
Jack Robison 0313f857ba don't try to delete the stream terminator from the blob table when deleting a stream 2018-03-15 17:44:54 -04:00
Jack Robison 8af96a0c51 don't try to announce the stream terminator 2018-03-15 17:42:21 -04:00
Thomas Zarebczan db821c51b7 update changelog 2018-03-15 10:15:02 -04:00
Jack Robison adb1291c09 fix blob list key error 2018-03-15 09:55:40 -04:00
Thomas Zarebczan b218e84d29 update channel balance checking to match claim
Simple balance check before calling get_max_usable_balance_for_claim - matches what happens on the claim.

Also provide clearer error message about maximum amount
2018-03-15 09:42:17 -04:00
Jack Robison 80fcb760b7 Bump version 0.19.1rc5 --> 0.19.1rc6
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-14 16:45:39 -04:00
Jack Robison 5ca48944c6 requirements 2018-03-14 16:45:08 -04:00
Jack Robison bc2f5901a7 Merge branch 'add-txlbryum' 2018-03-14 15:07:57 -04:00
Lex Berezhny 633b49da92 removed RequestCounter 2018-03-14 15:01:47 -04:00
Jack Robison 76b3d219c8 Bump version 0.19.1rc4 --> 0.19.1rc5
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-14 13:54:05 -04:00
Jack Robison e94d10d3eb only call get_max_usable_balance_for_claim in Wallet.py if the bid looks too high for the balance 2018-03-14 13:47:34 -04:00
Jack Robison 25eb64b346 Bump version 0.19.1rc3 --> 0.19.1rc4
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-13 17:34:08 -04:00
Jack Robison 9b44ba7c84 only call get_max_usable_balance when the bid might be too high 2018-03-13 17:32:29 -04:00
Jack Robison 69ecc6ce90 Bump version 0.19.1rc2 --> 0.19.1rc3
Signed-off-by: Jack Robison <jackrobison@lbry.io>
2018-03-13 11:44:27 -04:00
Jack Robison cde78f8134 migrate configured download directory instead of assuming the default 2018-03-13 11:40:34 -04:00
Alex Grintsvayg ed3e825472 cleaned up duplicates in changelog 2018-03-13 09:58:20 -04:00
Alex Grin 6dd8138c05 Merge pull request #1138 from lbryio/dht-compat
make dht forwards-compatible for upcoming switch to standardized bencode dict keys
2018-03-13 09:47:00 -04:00
Alex Grintsvayg d0aa0a7d0c make dht forwards-compatible for upcoming switch to standardized bencode dict keys 2018-03-13 09:43:32 -04:00
Lex Berezhny 37bfed8aee Merge pull request #1137 from lbryio/new-channel-fix
Error cleanly when claiming a new channel with exact or higher amount than balance
2018-03-12 16:17:18 -04:00
hackrush 16f2a5429b Name and output message fix. 2018-03-13 01:26:18 +05:30
Jack Robison e881b608bf Revert "don't block daemon startup on the file manager"
This reverts commit f25b772
2018-03-12 10:44:29 -04:00
hackrush f5aadf3918 Sanitized logged error for channel and name claims 2018-03-09 21:28:33 +05:30
hackrush 6d4af4ba90 Error cleanly when claiming a new channel with exact or higher amount than balance
fixes #1107
2018-03-09 21:28:33 +05:30
Jack Robison 2040748c62 add stratum client protocol 2018-03-06 14:30:19 -05:00
404 changed files with 29576 additions and 27535 deletions

No files matched your search

-44
View File
@@ -1,44 +0,0 @@
version: 1.0.{build}
environment:
GH_TOKEN:
secure: LiI5jyuHUw6XbH4kC3gP1HX4P/v4rwD/gCNtaFhQu2AvJz1/1wALkp5ECnIxRySN
key_pass:
secure: u6DydPcdrUJlxGL9uc7yQRYG8+5rY6aAEE9nfCSzFyNzZlX9NniOp8Uh5ZKQqX7bGEngLI6ipbLfiJvn0XFnhbn2iTkOuMqOXVJVOehvwlQ=
pfx_key:
secure: 1mwqyRy7hDqDjDK+TIAoaXyXzpNgwruFNA6TPkinUcVM7A+NLD33RQLnfnwVy+R5ovD2pUfhQ6+N0Fqebv6tZh436LIEsock+6IOdpgFwrg=
AWS_ACCESS_KEY_ID:
secure: iVGwoJ7ogspjSmuqr+haVPLglSgQsp6tUZx6mIlKH7Q=
AWS_SECRET_ACCESS_KEY:
secure: zKaqdZGPl0exDL5YhJkb33prSemC9Rzg9S7Lw2wFy1WnJ6ffgl6mQH7jqJDUTqsY
#notifications:
# - provider: Slack
# incoming_webhook:
# secure: LuxwG5OZnnA//gmSXzCKu8/FRqYjCgGfVFqajSsGHeQ1HQNp7rYNhQpsO8/3PK63xKJj3wzt86DJekf9q9Q5OcHa9AHXUQbEveX0psd7elw=
clone_folder: c:\projects\lbry
test_script:
- cd C:\projects\lbry\
- pip install cython
- pip install mock pylint unqlite
- pip install .
- pylint lbrynet
# disable tests for now so that appveyor can build the app
- set PYTHONPATH=.
- trial lbrynet.tests
# TODO: integration tests do not work
#- python -m unittest discover tests/integration
#- rvm use 2.3.1 && gem install danger --version '~> 4.0' && danger
build_script:
- cd C:\projects\lbry\build\
- ps: .\build.ps1
artifacts:
- path: build\dist\*.zip
name: lbrynet-daemon
+7
View File
@@ -0,0 +1,7 @@
.git
.tox
__pycache__
dist
lbrynet.egg-info
docs
tests
+22 -1
View File
@@ -4,7 +4,7 @@ Thanks for reporting an issue to LBRY and helping us improve!
To make it possible for us to help you, please fill out below information carefully.
Before reporting any issues, please make sure that you're using the latest version.
- App: https://github.com/lbryio/lbry-app/releases
- App: https://github.com/lbryio/lbry-desktop/releases
- Daemon: https://github.com/lbryio/lbry/releases
We are also available on Discord at https://chat.lbry.io
@@ -51,3 +51,24 @@ Tell us what happens instead
## Screenshots
<!-- If a screenshot would help explain the bug, please include one or two here -->
## Internal Use
### Acceptance Criteria
1.
2.
3.
### Definition of Done
- [ ] Tested against acceptance criteria
- [ ] Tested against the assumptions of user story
- [ ] The project builds without errors
- [ ] Unit tests are written and passing
- [ ] Tests on devices/browsers listed in the issue have passed
- [ ] QA performed & issues resolved
- [ ] Refactoring completed
- [ ] Any configuration or build changes documented
- [ ] Documentation updated
- [ ] Peer Code Review performed
+38
View File
@@ -0,0 +1,38 @@
## PR Checklist
Please check all that apply to this PR using "x":
- [ ] I have checked that this PR is not a duplicate of an existing PR (open, closed or merged)
- [ ] I have checked that this PR does not introduce a breaking change
- [ ] This PR introduces breaking changes and I have provided a detailed explanation below
## PR Type
What kind of change does this PR introduce?
Why is this change necessary?
<!-- Please check all that apply to this PR using "x". -->
- [ ] Bugfix
- [ ] Feature
- [ ] Breaking changes (bugfix or feature that introduces breaking changes)
- [ ] Code style update (formatting)
- [ ] Refactoring (no functional changes)
- [ ] Documentation changes
- [ ] Other - Please describe:
## Fixes
Issue Number: N/A
## What is the current behavior?
## What is the new behavior?
## Other information
<!-- If this PR contains a breaking change, please describe the impact and solution strategy for existing applications below. -->
+7 -23
View File
@@ -1,25 +1,9 @@
*.pyc
*.egg
*.so
*.xml
*.iml
*.log
*.pem
*.decTest
*.prof
.#*
/build
/dist
/.tox
/.idea
/.coverage
/build/build
/build/dist
/bulid/requirements_base.txt
/lbrynet.egg-info
/docs_build
/lbry-venv
.idea/
.coverage
.DS_Store
# temporary files from the twisted.trial test runner
lbrynet.egg-info
__pycache__
_trial_temp/
+11 -7
View File
@@ -9,7 +9,7 @@
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS
ignore=CVS,schema
# Add files or directories matching the regex patterns to the
# blacklist. The regex matches against base names, not paths.
@@ -62,7 +62,7 @@ confidence=
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# disable everything first and then re-enable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
@@ -121,7 +121,11 @@ disable=
unidiomatic-typecheck,
global-at-module-level,
inconsistent-return-statements,
keyword-arg-before-vararg
keyword-arg-before-vararg,
assignment-from-no-return,
useless-return,
assignment-from-none,
stop-iteration-return
[REPORTS]
@@ -350,13 +354,13 @@ ignore-mixin-members=yes
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=twisted.internet.reactor,leveldb,distutils
ignored-modules=leveldb,distutils
# Ignoring distutils because: https://github.com/PyCQA/pylint/issues/73
# List of classes names for which member attributes should not be checked
# (useful for classes with attributes dynamically set). This supports can work
# with qualified names.
ignored-classes=twisted.internet.reactor,RequestMessage
# ignored-classes=
# List of members which are set dynamically and missed by pylint inference
@@ -386,7 +390,7 @@ int-import-graph=
[DESIGN]
# Maximum number of arguments for function / method
max-args=5
max-args=10
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
@@ -405,7 +409,7 @@ max-branches=12
max-statements=50
# Maximum number of parents for a class (see R0901).
max-parents=7
max-parents=8
# Maximum number of attributes for a class (see R0902).
max-attributes=7
+132 -39
View File
@@ -1,47 +1,140 @@
os: linux
dist: trusty
sudo: required
dist: xenial
language: python
python: 2.7
python: "3.7"
branches:
except:
- gh-pages
jobs:
include:
notifications:
slack:
secure: "Am13HPtpgCMljh0MDVuoFHvQXB8yhf4Kvf/qAeSp5N0vsHGL70CSF9Ahccw8dVPE6mbuak1OGtSUb6/UaErLHkpz3ztaRLkDa9x7CmBB3Kynnh8oO2VbB7b/2ROULqkhF4VZmAnNfwrQrbC3gs8Sybp261Nyc7y4ww15xDYBrk2fyq4ds2DCaJdRxfJUJFonrZ6KXr3fVaXosO6cjuyS8eRodcmrqsT4cCtinjNTD1hGWoH107E4ObSmpVelxQO193KhNJMRiLlEcVkvYUOqIWBtwdGHbNE/6Yeuq1TXgKJ0KeJWAmW3wTfUYNngGXNAsyCnrhul5TKNevNzfIAQZHvRsczYiWPJV6LtohHT0CcUiCXJtvEPOyahEBfwK3etY/xxFqny7N9OEmpdW2sgsEPNPX2LJynJti2rQA9SuAD1ogR3ZpDy/NXoaAZf8PTdPcuNUMULV9PGG7tVrLBecO/W1qO6hdFxwlLdgqGLxAENZgGp++v/DhPk/WvtmHj7iTbRq0nxaTWyX5uKOn2ADH+k/yfutjv6BsQU9xNyPeZEEtuEpc6X6waiYn/8G9vl9PecvKC5H0MgsZ6asAxmg7mZ3VSMFG7mo8ENeOhSZ0Oz6ZTBILL3wFccZA9uJIq7NWmqC9dRiGiuKXBB62No7sINoHg3114e2xYa9qvNmGg="
- stage: code quality
name: "pylint lbrynet"
install:
- pip install astroid==2.0.4 aiohttp==3.4.4
# newer astroid and aiohttp fails in pylint so we pre-install older version
- pip install pylint
- pip install git+https://github.com/lbryio/torba.git#egg=torba
- pip install -e .
script: pylint lbrynet
- stage: test
name: "Unit Tests"
install:
- pip install coverage
- pip install git+https://github.com/lbryio/torba.git#egg=torba
- pip install -e .
script:
- HOME=/tmp coverage run -p --source=lbrynet -m unittest discover -vv tests.unit
after_success:
- coverage combine
- bash <(curl -s https://codecov.io/bash)
- name: "Integration Tests"
install:
- pip install tox-travis coverage
- pushd .. && git clone https://github.com/lbryio/torba.git && popd
script: tox
after_success:
- coverage combine tests/
- bash <(curl -s https://codecov.io/bash)
- stage: build
name: "Windows"
language: generic
services:
- docker
install:
- docker pull lbry/pyinstaller34_32bits:py371
script:
- python scripts/set_build.py
- docker run -v "$(pwd):/src/lbry" lbry/pyinstaller34_32bits:py371 lbry/scripts/wine_build.sh
- sudo zip -j dist/lbrynet-windows.zip dist/lbrynet.exe
deploy:
provider: releases
api_key: $GITHUB_OAUTH_TOKEN
file: dist/lbrynet-windows.zip
skip_cleanup: true
overwrite: true
draft: true
on:
tags: true
addons:
artifacts:
working_dir: dist
paths:
- lbrynet-windows.zip
target_paths:
- /daemon/build-${TRAVIS_BUILD_NUMBER}_commit-${TRAVIS_COMMIT:0:7}_branch-${TRAVIS_BRANCH}$([ ! -z ${TRAVIS_TAG} ] && echo _tag-${TRAVIS_TAG})
- &build
name: "Linux"
env: OS=linux
install:
- pip3 install pyinstaller
- pip3 install git+https://github.com/lbryio/torba.git
- python scripts/set_build.py
- pip3 install -e .
script:
- pyinstaller -F -n lbrynet lbrynet/extras/cli.py
- chmod +x dist/lbrynet
- zip -j dist/lbrynet-${OS}.zip dist/lbrynet
- ./dist/lbrynet --version
deploy:
provider: releases
api_key: $GITHUB_OAUTH_TOKEN
file: dist/lbrynet-${OS}.zip
skip_cleanup: true
overwrite: true
draft: true
on:
tags: true
addons:
artifacts:
working_dir: dist
paths:
- lbrynet-${OS}.zip
# artifact uploader thinks lbrynet is a directory, https://github.com/travis-ci/artifacts/issues/78
target_paths:
- /daemon/build-${TRAVIS_BUILD_NUMBER}_commit-${TRAVIS_COMMIT:0:7}_branch-${TRAVIS_BRANCH}$([ ! -z ${TRAVIS_TAG} ] && echo _tag-${TRAVIS_TAG})
- <<: *build
name: "Mac"
os: osx
osx_image: xcode8.3
language: generic
env: OS=mac
cache: false
before_install:
- brew upgrade python || true
- brew upgrade python || true
install:
- python3 --version
- pip3 --version
- pip3 install pyinstaller
- git clone https://github.com/lbryio/torba.git --depth 1
- sed -i -e "s/'plyvel',//" torba/setup.py
- cd torba && pip3 install -e . && cd ..
- python3 scripts/set_build.py
- pip3 install -e .
- if: tag IS present
stage: build
name: "Wallet Server Docker Image - Tagged Release"
script:
- set -e
- echo "$DOCKER_PASSWORD" | docker login --username "$DOCKER_USERNAME" --password-stdin
- travis_retry docker build -t lbry/wallet-server:$TRAVIS_TAG -f scripts/Dockerfile.wallet_server .
- docker push lbry/wallet-server:$TRAVIS_TAG
- if: tag IS blank AND branch = master
stage: build
name: "Wallet Server Docker Image - Master"
script:
- set -e
- echo "$DOCKER_PASSWORD" | docker login --username "$DOCKER_USERNAME" --password-stdin
- travis_retry docker build -t lbry/wallet-server:master -f scripts/Dockerfile.wallet_server .
- docker push lbry/wallet-server:master
cache:
directories:
- $HOME/.cache/pip
- $HOME/Library/Caches/pip
- $TRAVIS_BUILD_DIR/cache/wheel
addons:
#srcclr:
# debug: true
apt:
packages:
- libgmp3-dev
- build-essential
- git
- libssl-dev
- libffi-dev
before_install:
- virtualenv venv
- source venv/bin/activate
install:
- pip install -U pip
- pip install -r requirements.txt
- pip install .
script:
- pip install cython
- pip install mock pylint unqlite
- pylint lbrynet
- PYTHONPATH=. trial lbrynet.tests
- python -m unittest discover lbrynet/tests/integration -v
- rvm install ruby-2.3.1
- rvm use 2.3.1 && gem install danger --version '~> 4.0' && danger
- $TRAVIS_BUILD_DIR/.tox
+371 -120
View File
@@ -4,35 +4,364 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/) with
regard to the json-rpc api. As we're currently pre-1.0 release, we
can and probably will change functionality and break backwards compatability
can and probably will change functionality and break backwards compatibility
at anytime.
## [Unreleased]
## [0.30.1] - 2018-12-09
### API
* add `aioupnp_version` to `upnp` field in the response to `status`
* fixed uncaught claim signature errors
* `--blocking` support for `abandon` command (https://github.com/lbryio/lbry/pull/1618)
* don't create SSL keys if SSL is disabled (https://github.com/lbryio/lbry/pull/1644)
* added `account_send` command (https://github.com/lbryio/lbry/pull/1654)
* fixes for claim send to address (https://github.com/lbryio/lbry/pull/1651)
* better handle errors on resolve (https://github.com/lbryio/lbry/pull/1662)
* fix `transaction_show` (https://github.com/lbryio/lbry/pull/1689)
### Schema
* merged `lbryschema` repository into `lbrynet` (https://github.com/lbryio/lbry/pull/1444)
* updated `protobuf` to `3.2.0` (https://github.com/lbryio/lbry/issues/1602)
* use `types` repo for generating python protobufs (https://github.com/lbryio/lbry/pull/1605)
### Wallet
* fixed resolving names with invalid channel signatures
* fixed wallet sync issues
* fixed wallet locking/encryption issues
### DHT
* fixed retrying kademlia rpc calls (https://github.com/lbryio/lbry/pull/1495)
* fixed `KeyError` in `iterativeFindValue`, this is used by `peer_list` and when finding peers to download blobs from (https://github.com/lbryio/lbry/issues/1638)
* settings external ip address and ability to disable more DHT components (https://github.com/lbryio/lbry/pull/1604)
* improved routing table bucket initialization (https://github.com/lbryio/lbry/pull/1633)
### Database
* resolved too many variables in sqlite issue (https://github.com/lbryio/lbry/pull/1612)
### Misc
* switch to Python 3.7
* brought back icon for windows binary
* using custom list of mime types with more types (https://github.com/lbryio/lbry/pull/1700)
* add balance to startup report (https://github.com/lbryio/lbry/pull/1613)
* drop `six` dependency (https://github.com/lbryio/lbry/pull/1660)
* drop `mock` dependency (https://github.com/lbryio/lbry/pull/1661)
## [0.30.0] - 2018-10-22
This release is the most ambitious upgrade to lbrynet yet; including a brand
new wallet implementation, switch to Python 3 of the entire code base and
many changes to the API such as pagination and a more consistent output from
most commands.
### Security
*
*
* upgraded `cryptography` package.
* switch to `coincurve` for private / public key generation (addresses)
* support for connecting to daemon via SSL
* improved workflow for encrypt/decrypt of wallet
### API
* unified all command line executables into a single `lbrynet` executable.
* added `use_https` configuration setting (off by default), if true the api
server will generate a self signed ssl certificate when started and only
handle api requests on the https address (default of `https://localhost:5279/lbryapi`)
* deprecated `daemon_stop` command, use `stop` instead.
* deprecated `wallet_balance` command, use `account_balance` instead.
* deprecated `wallet_unlock` command, use `account_unlock` instead.
* deprecated `wallet_decrypt` command, use `account_decrypt` instead.
* deprecated `wallet_encrypt` command, use `account_encrypt` instead.
* deprecated `wallet_prefill_addresses` command, use `account_fund` instead.
* deprecated `wallet_list` command, use `address_list` instead.
* deprecated `wallet_is_address_mine` command, use `address_is_mine` instead.
* deprecated `wallet_public_key` command.
* deprecated `wallet_new_address` command.
* deprecated `wallet_unused_address` command, use `address_unused` instead.
* deprecated `claim_renew` command.
* added `account_list` command to list accounts including their balance.
* added `account_add` command to add a previously created account from seed or private key.
* added `account_create` command to generate a new account.
* added `account_remove` command to remove an account from wallet.
* added `account_set` command to change a setting on an account.
* added `account_balance` command to get just the account balance.
* added `account_unlock` command to unlock an encrypted account.
* added `account_lock` command to lock an encrypted account.
* added `account_encrypt` command to encrypt an account.
* added `account_decrypt` command to decrypt an account.
* added `account_fund` command to move funds between or within an account in various ways.
* added `account_max_address_gap` command to find large gaps of unused addresses.
* added `address_list` command to list addresses.
* added `address_is_mine` command to check if an address is one of your addresses.
* added `address_unused` command to get existing or generate a new unused address.
* added pagination support for `address_list`, `channel_list`, `claim_list_mine`,
`transaction_list` and `utxo_list`.
* added `upnp` field to `status` response
* removed `send_amount_to_address` command previously marked as deprecated
* removed `channel_list_mine` command previously marked as deprecated
* removed `get_availability` command previously marked as deprecated
* updated `txupnp` dependency to `aioupnp` (UPnP for asyncio)
### Wallet
* changed to a new wallet implementation: [torba](https://github.com/lbryio/torba).
* changed wallet file format to support multiple accounts in one wallet.
* moved transaction data from wallet file into an sqlite database.
* changed channel certificates to be keyed by txid:nout instead of claim_id which
makes it possible to recover old certificates.
### File Manager
* Extensive internal changes as a result of porting to Python 3.
### DHT
* Extensive internal changes as a result of porting to Python 3.
### P2P
* Extensive internal changes as a result of porting to Python 3.
### Reflector
* No major changes to reflector.
### Database
* No changes to existing storage module.
### Known Issues
* Channel certificate import/export is currently not working.
## [0.21.2] - 2018-08-23
### Fixed
* fixed the inconsistencies in API and CLI docstrings
* `blob_announce` error when announcing a single blob
* `blob_list` error when looking up blobs by stream or sd hash
* issue in dht ping queue where enqueued pings that aren't yet due wouldn't be rescheduled
* blob mirror downloader not finishing streams that were partially uploaded at the time of the download attempt (https://github.com/lbryio/lbry/issues/1376)
### Deprecated
* `report_bug` jsonrpc command
*
### Changed
* reflector server to periodically check and set `should_announce` for sd and head blobs instead of during each request
* reflector server to use `SQLiteStorage` to find needed blob hashes for a stream
* file manager startup to not block daemon startup
## [0.21.1] - 2018-08-13
### Fixed
* `download_progress` field in `blockchain_headers` section of `status` not initializing correctly when resuming a download (https://github.com/lbryio/lbry/issues/1355)
* `wallet_send` not accepting decimal amounts (https://github.com/lbryio/lbry/issues/1356 commit https://github.com/lbryio/lbry/commit/1098ca0494ece420c70fc57f69d6d388715a99b8)
### Added
* scripts to autogenerate documentation
*
* `is_locked` to `wallet` in `status` response (https://github.com/lbryio/lbry/issues/1354, commit https://github.com/lbryio/lbry/commit/153022a1a7122ab6d31d3db433dccbe469bbcb3c)
### Changed
* Bumped `lbryum` requirement to 3.2.4 [see changelog](https://github.com/lbryio/lbryum/blob/master/CHANGELOG.md#324---2018-08-13)
## [0.21.0] - 2018-08-09
### Fixed
* check `claim_address` and `change_address` earlier on publishing, to avoid hard to understand errors later in the process (pr https://github.com/lbryio/lbry/pull/1347)
* loggly error reporting not following `share_usage_data` (pr https://github.com/lbryio/lbry/pull/1328)
* improper error handling when data is not valid JSON (pr https://github.com/lbryio/lbry/pull/1326)
* blob mirroring being set in unrelated tests, making them fail (pr https://github.com/lbryio/lbry/pull/1348)
* http blob mirroring edge cases (pr https://github.com/lbryio/lbry/pull/1315)
* external ports in log messages not showing the correct external port from the upnp redirects (https://github.com/lbryio/lbry/issues/1338) (pr https://github.com/lbryio/lbry/pull/1349)
* miniupnpc fallback issues in txupnp (https://github.com/lbryio/lbry/issues/1341) (pr https://github.com/lbryio/lbry/pull/1349)
* upnp error when disabled on router and a non-gateway is found, such as chromecast (https://github.com/lbryio/lbry/issues/1352) (https://github.com/lbryio/lbry/commit/dca4af942fbe95547794213775f0a62cd04a393f)
### Deprecated
* automatic claim renew, this is no longer needed
### Changed
* api server class to use components, and for all JSONRPC API commands to be callable so long as the required components are available. (pr https://github.com/lbryio/lbry/pull/1294)
* return error messages when required conditions on components are not met for API calls (pr https://github.com/lbryio/lbry/pull/1328)
* `status` to no longer return a base58 encoded `lbry_id`, instead return this as the hex encoded `node_id` in a new `dht` field. (pr https://github.com/lbryio/lbry/pull/1328)
* `startup_status` field in the response to `status` to be a dict of component names to status booleans (pr https://github.com/lbryio/lbry/pull/1328)
* renamed the `blockchain_status` field in the response to `status` to `wallet` (pr https://github.com/lbryio/lbry/pull/1328)
* moved and renamed `wallet_is_encrypted` to `is_encrypted` in the `wallet` field in the response to `status` (pr https://github.com/lbryio/lbry/pull/1328)
* moved wallet, upnp and dht startup code from `Session` to `Components` (pr https://github.com/lbryio/lbry/pull/1328)
* attempt blob downloads from http mirror sources (by default) concurrently to p2p sources (pr https://github.com/lbryio/lbry/pull/1233)
* replace miniupnpc with [txupnp](https://github.com/lbryio/txupnp). Since txupnp is still under development, it will internally fall back to miniupnpc. (pr https://github.com/lbryio/lbry/pull/1328)
* simplified test_misc.py in the functional tests (pr https://github.com/lbryio/lbry/pull/1328)
* update `cryptography` requirement to 2.3 (pr https://github.com/lbryio/lbry/pull/1333)
### Added
* `skipped_components` list to the response from `status` (pr https://github.com/lbryio/lbry/pull/1328)
* component statuses (`blockchain_headers`, `dht`, `wallet`, `blob_manager` `hash_announcer`, and `file_manager`) to the response to `status` (pr https://github.com/lbryio/lbry/pull/1328)
* `skipped_components` config setting, accepts a list of names of components to not run (pr https://github.com/lbryio/lbry/pull/1294)
* `ComponentManager` for managing the life-cycles of dependencies (pr https://github.com/lbryio/lbry/pull/1294)
* `requires` decorator to register the components required by a `jsonrpc_` command, to facilitate commands registering asynchronously (pr https://github.com/lbryio/lbry/pull/1294)
* unit tests for `ComponentManager` (pr https://github.com/lbryio/lbry/pull/1294)
* script to generate docs/api.json file (https://github.com/lbryio/lbry.tech/issues/42)
* additional information to the balance error message when editing a claim (pr https://github.com/lbryio/lbry/pull/1309)
* `address` and `port` arguments to `peer_ping` (https://github.com/lbryio/lbry/issues/1313) (pr https://github.com/lbryio/lbry/pull/1299)
* ability to download from HTTP mirrors by setting `download_mirrors` (prs https://github.com/lbryio/lbry/pull/1233 and https://github.com/lbryio/lbry/pull/1315)
* ability to filter peers from an iterative find value operation (finding peers for a blob). This is used to filter peers we've already found for a blob when accumulating the list of peers. (pr https://github.com/lbryio/lbry/pull/1287)
### Removed
* short(single dashed) arguments for `lbrynet-cli`
*
* `session_status` argument and response field from `status` (pr https://github.com/lbryio/lbry/pull/1328)
* most of the internal attributes from `Daemon` (pr https://github.com/lbryio/lbry/pull/1294)
## [0.20.4] - 2018-07-18
### Fixed
* spelling errors in messages printed by `lbrynet-cli`
* high CPU usage when a stream is incomplete and the peers we're requesting from have no more blobs to send us (https://github.com/lbryio/lbry/pull/1301)
### Changed
* keep track of failures for DHT peers for up to ten minutes instead of indefinitely (https://github.com/lbryio/lbry/pull/1300)
* skip ignored peers from iterative lookups instead of blocking the peer who returned them to us too (https://github.com/lbryio/lbry/pull/1300)
* if a node becomes ignored during an iterative find cycle remove it from the shortlist so that we can't return it as a result nor try to probe it anyway (https://github.com/lbryio/lbry/pull/1303)
## [0.20.3] - 2018-07-03
### Fixed
* `blob_list` raising an error when blobs in a stream haven't yet been created (8a0d0b44ddf9cbeb2a9074eed39d6064ce21df64)
* stopping a download potentially raising an attribute error (https://github.com/lbryio/lbry/pull/1269)
* file manager startup locking up when there are many files for some channels (https://github.com/lbryio/lbry/pull/1281)
* improper sorting when getting the closest peers to a hash (https://github.com/lbryio/lbry/pull/1282)
### Changed
* raised the default `peer_search_timeout` setting from 3 to 30 and added logging for when it happens (https://github.com/lbryio/lbry/pull/1283)
* change iterative find stop condition on find value to allow it to continue until a value is found or it times out (https://github.com/lbryio/lbry/pull/1283)
* include all of our own blobs in the local dht datastore (as if we had announced them to ourselves) (https://github.com/lbryio/lbry/pull/1280)
* ignore dht `store` token validation errors for the first expiration-time after startup (fixes failed `store` requests after a restart) (https://github.com/lbryio/lbry/pull/1280)
### Removed
* `jsonrpclib` as a requirement for the project (https://github.com/lbryio/lbry/pull/1274)
## [0.20.2] - 2018-06-23
### Changed
* Bumped `lbryschema` requirement to 0.0.16 [see changelog](https://github.com/lbryio/lbryschema/blob/master/CHANGELOG.md#0016---2018-06-23)
* Bumped `lbryum` requirement to 3.2.3 [see changelog](https://github.com/lbryio/lbryum/blob/master/CHANGELOG.md#323---2018-06-23)
* save claims to sqlite in batches to speed up `resolve` queries for many uris
## [0.20.1] - 2018-06-19
### Fixed
* fixed token validation error when the dht node has just been started (https://github.com/lbryio/lbry/issues/1248)
* fixed a race condition when inserting a blob into the database (https://github.com/lbryio/lbry/issues/1129)
* reflector server incorrectly responding as if it has all the blobs for a stream that was only partially uploaded to it
* `publish` raising a database error when updating a claim that we don't have a file for (https://github.com/lbryio/lbry/issues/1165)
* blob client protocol not tearing itself down properly after a failure (https://github.com/lbryio/lbry/issues/950)
* lockup in wallet startup when one or more lbryumx servers are unavailable (https://github.com/lbryio/lbry/issues/1245)
* download being stopped if the sd blob downloaded and data did not start within the timeout (https://github.com/lbryio/lbry/issues/1172)
### Changed
* Bumped `lbryum` requirement to 3.2.2 [see changelog](https://github.com/lbryio/lbryum/blob/master/CHANGELOG.md#322---2018-06-19)
* `publish` to accept bid as a decimal string
## [0.20.0] - 2018-06-13
### TL;DR
This milestone release includes a large number of bug fixes, changes and additions covering all aspects of the daemon. Notable improvements include:
* Faster and more reliable downloading and uploading of content resulting from substantial work done on the Distributed Hash Table algorithms and protocol.
* Faster blockchain synchronization (headers) by downloading them from Amazon S3 under certain conditions.
* Faster overall app startup due to better optimized SQL queries.
* Power users of the `lbrynet-cli` will benefit from many bug fixes to commonly used commands and improvements in sorting of output.
* Please review the full change log for more details on specific fixes, changes and additions.
### Fixed
* fix payment rate manager typo ([1236](https://github.com/lbryio/lbry/pull/1236))
* handling error from dht clients with old `ping` method
* blobs not being re-announced if no peers successfully stored, now failed announcements are re-queued
* issue where an `AuthAPIClient` (used by `lbrynet-cli`) would fail to update its session secret and keep making new auth sessions, with every other request failing
* `use_auth_http` in a config file being overridden by the default command line argument to `lbrynet-daemon`, now the command line value will only override the config file value if it is provided
* `lbrynet-cli` not automatically switching to the authenticated client if the server is detected to be using authentication. This resulted in `lbrynet-cli` failing to run when `lbrynet-daemon` was run with the `--http-auth` flag
* fixed error when using `claim_show` with `txid` and `nout` arguments
* fixed error when saving server list to conf file ([1209](https://github.com/lbryio/lbry/pull/1209))
### Changed
* if the `use_authentication` setting is configured, use authentication for all api methods instead of only those with the `auth_required` decorator
* regenerate api keys on startup if the using authentication
* support both positional and keyword args for api calls
* `blob_announce` to queue the blob announcement but not block on it
* `peer_list` to return a list of dictionaries instead of a list of lists, added peer node ids to the results
* predictable result sorting for `claim_list` and `claim_list_mine` ([1216](https://github.com/lbryio/lbry/pull/1216) and [1208](https://github.com/lbryio/lbry/pull/1208))
* increase the default `auto_re_reflect_interval` setting to a day and the default `concurrent_announcers` setting to 10
* download blockchain headers from s3 before starting the wallet when the local height is more than `s3_headers_depth` (a config setting) blocks behind ([1177](https://github.com/lbryio/lbry/pull/1177))
* check headers file integrity on startup, removing/truncating the file to force re-download when necessary
* support partial headers file download from S3 ([1189](https://github.com/lbryio/lbry/pull/1189))
* refactor `add_completed_blobs` on storage.py, simplifying into less queries ([1226](https://github.com/lbryio/lbry/pull/1226))
* full verification of streams only during database migration instead of every startup ([1195](https://github.com/lbryio/lbry/pull/1195))
* database batching functions for starting up the file manager
* added `single_announce` and `last_announced_time` columns to the `blob` table in sqlite
* track successful reflector uploads in sqlite to minimize how many streams are attempted by auto re-reflect ([1194](https://github.com/lbryio/lbry/pull/1194))
* pass the sd hash to reflector ClientFactory instead of looking it up from the database
* dht logging to be more verbose with errors and warnings
* `store` kademlia rpc method to block on the call finishing and to return storing peer information
* kademlia protocol to minimally delay writes to the UDP socket
* several internal dht functions to use inlineCallbacks
* `DHTHashAnnouncer` and `Node` manage functions to use `LoopingCall`s instead of scheduling with `callLater`.
* refactored `DHTHashAnnouncer` to no longer use locks, use a `DeferredSemaphore` to limit concurrent announcers
* decoupled `DiskBlobManager` from `DHTHashAnnouncer`, get blob hashes to announce from `SQLiteStorage`. The blob manager no longer announces blobs after they are completed, the hash announcer takes care of this now.
* changed the bucket splitting condition in the dht routing table to be more aggressive
* ping dht nodes who have stored to us periodically to determine whether we should include them as an active peer for the hash when we are queried. Nodes that are known to be not reachable by the node storing the record are no longer returned as peers by the storing node.
* changed dht bootstrap join process to better populate the routing table initially
* cache dht node tokens used during announcement to minimize the number of requests that are needed
* implement BEP0005 dht rules to classify nodes as good, bad, or unknown and for when to add them to the routing table (http://www.bittorrent.org/beps/bep_0005.html)
* refactored internal dht contact class to track failure counts/times, the time the contact last replied to us, and the time the node last requested something fom us ([1211](https://github.com/lbryio/lbry/pull/1211))
* refactored dht iterativeFind
* sort dht contacts returned by `findCloseNodes` in the routing table
* `reactor` and `callLater`, `listenUDP`, and `resolve` functions to be configurable (to allow easier testing)
* calls to get the current time to use `reactor.seconds` (to control callLater and LoopingCall timing in tests)
* temporarily disabled data price negotiation, treat all data as free
* disabled Cryptonator price feed
* use `treq` instead of `txrequests` ([1191](https://github.com/lbryio/lbry/pull/1191))
* updated `cryptography` version to 2.2.2
* removed `pycrypto` dependency, replacing all calls to `cryptography`
### Added
* `peer_ping` command
* `--sort` option in `file_list` ([1174](https://github.com/lbryio/lbry/pull/1174))
* `port` field to contacts returned by `routing_table_get`
* configurable `concurrent_announcers` and `s3_headers_depth` settings
* virtual kademlia network and mock udp transport for dht integration tests
* functional tests for bootstrapping the dht, announcing and expiring hashes, finding and pinging nodes, protocol version 0/1 backwards/forwards compatibility, and rejoining the network
* linux distro and desktop name added to analytics ([1218](https://github.com/lbryio/lbry/pull/1218))
* certifi module for Twisted SSL verification on Windows ([1213](https://github.com/lbryio/lbry/pull/1213))
* protocol version to dht requests and to the response from `findValue`
### Removed
* `announce_all` argument from `blob_announce`
* old `blob_announce_all` command
* unused `--wallet` argument to `lbrynet-daemon`, which used to be to support `PTCWallet`.
* `AuthJSONRPCServer.auth_required` decorator ([1161](https://github.com/lbryio/lbry/pull/1161))
* `OptimizedTreeRoutingTable` class used by the dht node for the time being
## [0.19.3] - 2018-05-04
### Changed
* download blockchain headers from s3 before starting the wallet when the local height is more than s3_headers_depth (a config setting) blocks behind (https://github.com/lbryio/lbry/pull/1177)
* un-deprecated report_bug command (https://github.com/lbryio/lbry/commit/f8e418fb4448a3ed1531657f8b3c608fb568af85)
## [0.19.2] - 2018-03-28
### Fixed
* incorrectly raised download cancelled error for already verified blob files
* infinite loop where reflector client keeps trying to send failing blobs, which may be failing because they are invalid and thus will never be successfully received
* docstring bugs for `stream_availability`, `channel_import`, and `blob_announce`
### Added
* `blob_reflect` command to send specific blobs to a reflector server
* unit test for docopt
### Removed
* `flags` decorator from server.py as short flags are no longer used when using api/cli methods
### Changed
* Bumped `lbryum` requirement to 3.2.1 [see changelog](https://github.com/lbryio/lbryum/blob/master/CHANGELOG.md#321---2018-03-28)
## [0.19.1] - 2018-03-20
### Fixed
* Fixed the inconsistencies in API and CLI docstrings
* `blob_announce` error when announcing a single blob
* `blob_list` error when looking up blobs by stream or sd hash ([1126](https://github.com/lbryio/lbry/pull/1126))
* Claiming a channel with the exact amount present in wallet would return a confusing error ([1107](https://github.com/lbryio/lbry/issues/1107))
* Channel creation to use same bid logic as for claims ([1148](https://github.com/lbryio/lbry/pull/1148))
### Deprecated
* `report_bug` jsonrpc command
### Changed
* Bumped `lbryschema` requirement to 0.0.15 [see changelog](https://github.com/lbryio/lbryschema/blob/master/CHANGELOG.md#0015---2018-03-20)
* Bumped `lbryum` requirement to 3.2.0 [see changelog](https://github.com/lbryio/lbryum/blob/master/CHANGELOG.md#320---2018-03-20)
* Reflector server to periodically check and set `should_announce` for sd and head blobs instead of during each request
* Reflector server to use `SQLiteStorage` to find needed blob hashes for a stream
### Added
* Scripts to auto-generate documentation ([1128](https://github.com/lbryio/lbry/pull/1128))
* Now updating new channel also takes into consideration the original bid amount, so now channel could be updated for wallet balance + the original bid amount ([1137](https://github.com/lbryio/lbry/pull/1137))
* Forward-compatibility for upcoming DHT bencoding changes
### Removed
* Short(single dashed) arguments for `lbrynet-cli`
## [0.19.0] - 2018-03-02
@@ -79,7 +408,6 @@ at anytime.
* manager classes to use new `SQLiteStorage` for database interaction. This class uses a single `lbrynet.sqlite` database file.
### Added
* link to instructions on how to change the default peer port
* `lbrynet-console`, a tool to run or connect to lbrynet-daemon and launch an interactive python console with the api functions built in.
* `--conf` CLI flag to specify an alternate config file
* `peer_port`, `disable_max_key_fee`, `auto_renew_claim_height_delta`, `blockchain_name`, and `lbryum_servers` to configurable settings
@@ -158,7 +486,7 @@ at anytime.
* Fixed https://github.com/lbryio/lbry/issues/923
* Fixed concurrent reflects opening too many files
* Fixed cases when reflecting would fail on error conditions
* Fixed deadlocks from occuring during blob writes
* Fixed deadlocks from occurring during blob writes
* Fixed and updated`lbrynet.tests.dht`
* Fixed redundant dht id
* Fixed dht `ping` method
@@ -181,7 +509,6 @@ at anytime.
### Added
* Added WAL pragma to sqlite3
* Added unit tests for `BlobFile`
* Updated exchange rate tests for the lbry.io api
* Use `hashlib` for sha384 instead of `pycrypto`
* Use `cryptography` instead of `pycrypto` for blob encryption and decryption
* Use `cryptography` for PKCS7 instead of doing it manually
@@ -256,7 +583,6 @@ at anytime.
## [0.15.0] - 2017-08-15
### Fixed
* Fixed reflector server blocking the `received_blob` reply on the server announcing the blob to the dht
* Fixed incorrect formatting of "amount" fields
* Fixed handling of SIGINT, SIGTERM.
* Fixed shutdown sequence
* Fix error when resolving an integer
@@ -380,7 +706,6 @@ at anytime.
### Fixed
* Race condition from improper initialization and shutdown of the blob manager database
* Various fixes for GetStream class used in API command get
* Download analytics error
* Fixed flag options in file_delete API command
@@ -440,39 +765,6 @@ at anytime.
## [0.10.3production4] - 2017-06-01
### Added
* Prevent publish of files with size 0
* Add `dht_status` parameter to `status` to include bandwidth and peer info
* Positional and flag arguments in lbrynet-cli
### Changed
* Changed keyword arguments in lbrynet-cli to use a -- prefix
* Using the help function in lbrynet-cli no longer requires lbrynet-daemon to be running
## [0.10.3production3] - 2017-05-30
### Changed
* Update `publish` to use {'currency': ..., 'address': ..., 'amount'} format for fee parameter, previously used old {currency: {'address': ..., 'amount': ...}} format
### Fixed
* Allow claim_show to be used without specifying name
* Fix licenseUrl field in `publish`
## [0.10.3production2] - 2017-05-30
### Fixed
* Allow claim_show to be used without specifying name
## [0.10.3] - 2017-05-23
### Added
@@ -481,8 +773,6 @@ at anytime.
### Changed
* Added optional `address` and `include_unconfirmed` params to `jsonrpc_wallet_balance` method
* Wait for subscriptions before announcing wallet has finished starting
* Cache claims in wallet storage for use looking claims up by id or outpoint
* Try to use cached claim info for `file_list`
* Convert wallet storage to inlinecallbacks
@@ -583,55 +873,38 @@ at anytime.
* Removed check_pending logic from Daemon
* Switched to txrequests so requests can use twisted event loop
* Renamed API command file_seed to file_set_status
* Don't add expected payment to wallet when payment rate is 0
### Fixed
* Fix restart procedure in DaemonControl
* Create download directory if it doesn't exist
* Fixed descriptor_get
* Fixed jsonrpc_reflect()
* Fixed api help return
* Fixed API command descriptor_get
* Fixed API command transaction_show
* Fixed error handling for jsonrpc commands
## [0.9.2rc1] - 2017-03-21
### Added
* Add `wallet_list` command
### Changed
* Dont add expected payment to wallet when payment rate is 0
### Fixed
* Fixed descriptor_get
* Fixed jsonrpc_reflect()
* Fixed api help return
* Fixed API command descriptor_get
* Fixed API command transaction_show
* Handle failure to decode claim cache file
## [0.9.1] - 2017-03-17
### Fixed
* Fix wallet_public_key API command
## [0.9.1rc5] - 2017-03-16
## [0.9.1] - 2017-03-17
### Added
* publish API command can take metadata fields as arguments
* Added `reflect_uploads` config to disable reflecting on upload
* Added `--version` flag
### Fixed
* Fix wallet_public_key API command
* Fixed jsonrpc_reflect()
* Fixed api help return
## [0.9.1rc2] - 2017-03-15
### Added
* Added `--version` flag
### Changed
* Removed `simplejson` dependency in favor of bulitin `json`
## [0.9.0rc17] - 2017-03-10
### Fixed
* Added string comparison to ClaimOutpoint (needed to look things up by outpoint)
* Remove unused API commands from daemon
* Fix file filter `outpoint`
* Made dictionary key names in API commmand outputs to be more consistent
## [0.9.0rc15] - 2017-03-09
* Made dictionary key names in API command outputs to be more consistent
### Added
* Add file filters: `claim_id`, `outpoint`, and `rowid`
* Make loggly logs less verbose
@@ -641,10 +914,7 @@ at anytime.
* Remove deprecated file commands: `get_lbry_files`, `get_lbry_file`, and `file_get`
* Remove deprecated `delete_lbry_file` command
* Return standard file json from `get`
### Fixed
* Added string comparison to ClaimOutpoint (needed to look things up by outpoint)
* Remove unused API commands from daemon
* Fix file filter `outpoint`
## [0.9.0rc12] - 2017-03-06
### Fixed
@@ -666,8 +936,6 @@ at anytime.
## [0.8.7] - 2017-02-21
## [0.8.6] - 2017-02-19
## [0.8.6rc0] - 2017-02-19
### Changed
* Add `file_get` by stream hash
* Add utils.call_later to replace reactor.callLater
@@ -677,14 +945,10 @@ at anytime.
* Fix sd blob timeout handling in `get_availability`, return 0.0
## [0.8.5] - 2017-02-18
## [0.8.5rc0] - 2017-02-18
### Fixed
* Fix result expected by ui from file_get for missing files
## [0.8.4] - 2017-02-17
## [0.8.4rc0] - 2017-02-17
### Changed
* Remove unused upload_allowed option
* Remove code related to packaging as that step is now done in the electron client
@@ -692,36 +956,32 @@ at anytime.
* Include download url in version check
### Fixed
* add misssing traceback to logging
* add missing traceback to logging
## [0.8.3] - 2017-02-15
### Fixed
* Get lbry files with pending claims
* Add better logging to help track down [#478](https://github.com/lbryio/lbry/issues/478)
* Catch UnknownNameErrors when resolving a name. [#479](https://github.com/lbryio/lbry/issues/479)
* Change EWOULDBLOCK error in DHT to warning. #481
* mark peers as down if it fails download protocol
* Made hash reannounce time to be adjustable to fix [#432](https://github.com/lbryio/lbry/issues/432)
* Fix recursion depth error upon failed blob
* Call stopProducing in reflector client file_sender when uploading is done
* Ensure streams in stream_info_manager are saved in lbry_file_manager
* Fixed file_delete not deleting data from stream_info_manager [#470](https://github.com/lbryio/lbry/issues/470)
* Fixed upload of bug reports to Slack ([#472](https://github.com/lbryio/lbry/issues/472))
* Fixed claim updates [#473](https://github.com/lbryio/lbry/issues/473)
* Handle ConnectionLost error in reflector client
* Fix updating a claim where the stream doesn't change
* Fix claim_abandon
### Changed
* Add blob_get, descriptor_get, and blob_delete
* Add filter keyword args to blob_list
* Refactor get_availability
* Add optional peer search timeout, add peer_search_timeout setting
## [0.8.3rc3] - 2017-02-14
## [0.8.3rc2] - 2017-02-13
## [0.8.3rc1] - 2017-02-13
### Changed
* make connection manager unit testeable
### Fixed
* Change EWOULDBLOCK error in DHT to warning. #481
* mark peers as down if it fails download protocol
* Made hash reannounce time to be adjustable to fix [#432](https://github.com/lbryio/lbry/issues/432)
## [0.8.3rc0] - 2017-02-10
### Changed
* Convert EncryptedFileDownloader to inlineCallbacks
* Convert EncryptedFileManager to use inlineCallbacks
* Convert Daemon._delete_lbry_file to inlineCallbacks
@@ -732,16 +992,7 @@ at anytime.
* Change callback condition in GetStream to the first data blob completing
* Add local and remote heights to blockchain status
### Fixed
* Fix recursion depth error upon failed blob
* Call stopProducing in reflector client file_sender when uploading is done
* Ensure streams in stream_info_manager are saved in lbry_file_manager
* Fixed file_delete not deleting data from stream_info_manager [#470](https://github.com/lbryio/lbry/issues/470)
* Fixed upload of bug reports to Slack ([#472](https://github.com/lbryio/lbry/issues/472))
* Fixed claim updates [#473](https://github.com/lbryio/lbry/issues/473)
* Handle ConnectionLost error in reflector client
* Fix updating a claim where the stream doesn't change
* Fix claim_abandon
## [0.8.1] - 2017-02-01
### Changed
+37 -64
View File
@@ -1,70 +1,48 @@
# Installing LBRY
The recommended way to install LBRY is to use an installer. We provide installers for all major operating systems. See the [README](README.md).
If only the JSON-RPC API server is needed, the recommended way to install LBRY is to use a pre-built binary. We provide binaries for all major operating systems. See the [README](README.md).
These instructions are for installing from source code, which is recommended if you are interested in doing development work or LBRY is not available on your operating system (godspeed, TempleOS users).
These instructions are for installing LBRY from source, which is recommended if you are interested in doing development work or LBRY is not available on your operating system (godspeed, TempleOS users).
Here's a video walkthrough of this setup, which is itself hosted by the LBRY network and provided via [spee.ch](https://github.com/lbryio/spee.ch):
[![Setup for development](https://spee.ch/2018-10-04-17-13-54-017046806.png)](https://spee.ch/967f99344308f1e90f0620d91b6c93e4dfb240e0/lbrynet-dev-setup.mp4)
## Prerequisites
Before starting, you'll need to install some additional libraries depending on your operating system.
Running `lbrynet` from source requires Python 3.6 or higher (3.7 is preferred). Get the installer for your OS [here](https://www.python.org/downloads/release/python-370/)
### Python
After installing python 3, you'll need to install some additional libraries depending on your operating system.
LBRY requires Python 2.7. Python 3 is not supported.
### Virtualenv
Once python 3 is installed run `python3 -m pip install virtualenv` to install virtualenv.
### Windows
Windows users will need to install `Visual C++ Build Tools`, which can be installed by [Microsoft Build Tools](Microsoft Build Tools 2015)
### OS X Prerequisites
### macOS
Install [xcode command line tools](https://developer.xamarin.com/guides/testcloud/calabash/configuring/osx/install-xcode-command-line-tools/) and [homebrew](http://brew.sh/).
macOS users will need to install [xcode command line tools](https://developer.xamarin.com/guides/testcloud/calabash/configuring/osx/install-xcode-command-line-tools/) and [homebrew](http://brew.sh/).
Remaining dependencies can then be installed by running:
```
brew install mpfr libmpc protobuf
sudo easy_install pip
sudo pip install virtualenv
brew install python3 protobuf
```
### Linux Prerequisites
### Linux
On Ubuntu (we recommend 16.04), install the following:
On Ubuntu (we recommend 18.04), install the following:
```
sudo apt-get install build-essential python2.7 python2.7-dev python-pip git python-virtualenv libssl-dev libffi-dev python-protobuf
sudo apt-get install build-essential python3.7 python3.7-dev git python-virtualenv libssl-dev python-protobuf
```
On Raspbian, you will also need to install `python-pyparsing`.
If you're running another Linux flavor, install the equivalent of the above packages for your system.
### Windows Prerequisites
Install [mingw32](http://www.mingw.org/) base and c++ compiler.
Add C:\MinGW\bin to the windows PATH.
Enable distutils to compile with mingw32 by creating a distutils.cfg file in *PYTHONPATH\Lib\distutils* containing:
```
[build]
compiler = mingw32
```
If using virtualenv, which is recommended, copy the *PYTHONPATH\Lib\distutils* directory to the virtualenv.
It's recommended to use [Unofficial Windows Binaries for Python Extension Packages](http://www.lfd.uci.edu/~gohlke/pythonlibs/) for as many of the required packages as possible.
Currently, available binaries include:
- Cython
- Twisted
- Zope.interface
- pywin32
- requests
Install the above binaries with `pip install *.whl`
Install pywin32 system files by run `python.exe Scripts\pywin32_postinstall.py -install` from an elevated command prompt.
Finally, you'll need [miniupnpc](https://pypi.python.org/pypi/miniupnpc/1.9). Download the source and compile with MinGW by running `mingw32make.bat`. Then install the module by running `python setupmingw32.py install`.
If you're running another Linux distro, install the equivalent of the above packages for your system.
## Installation
@@ -73,32 +51,27 @@ To install:
```
git clone https://github.com/lbryio/lbry.git
cd lbry
virtualenv lbry-venv
virtualenv lbry-venv --python=python3.7
source lbry-venv/bin/activate
python --version # Python 3 is not supported. Make sure you're on Python 2.7
pip install -U -r requirements.txt
pip install --editable .
python --version # Python 2 is not supported. Make sure you're on Python 3.7
pip install -e .
```
To start LBRY, run `lbrynet-daemon` in a terminal.
To verify your installation, `which lbrynet` should return a path inside of the `lbry-venv` folder created by the `virtualenv` command.
To verify your install, `which lbrynet-daemon` should return a path inside of the `lbry-venv` folder created by the `virtualenv` command.
## Run the tests
To run the unit tests from the repo directory:
```
trial --reactor=asyncio tests.unit
```
## Making a Change
## Usage
If you are not already inside the virtual environment, run:
To start the API server:
`lbrynet start`
```
source lbry-venv/bin/activate
```
Then run:
```
pip install --editable .
```
This will update `lbrynet-daemon` and other executables.
Happy hacking!
+38 -34
View File
@@ -1,52 +1,56 @@
[![Build Status](https://travis-ci.org/lbryio/lbry.svg?branch=master)](https://travis-ci.org/lbryio/lbry)
[![Coverage Status](https://coveralls.io/repos/github/lbryio/lbry/badge.svg)](https://coveralls.io/github/lbryio/lbry)
# <img src="https://raw.githubusercontent.com/lbryio/lbry/master/lbry.png" alt="LBRY" width="48" height="36" /> LBRY SDK [![Build Status](https://travis-ci.org/lbryio/lbry.svg?branch=master)](https://travis-ci.org/lbryio/lbry) [![Test Coverage](https://codecov.io/gh/lbryio/lbry/branch/master/graph/badge.svg)](https://codecov.io/gh/lbryio/lbry)
# LBRY
LBRY is a decentralized peer-to-peer network providing distribution, discovery, and purchase of digital content (data). It utilizes the [LBRY blockchain](https://github.com/lbryio/lbrycrd) as a global namespace and database of digital content. Blockchain entries contain searchable content metadata, identities, rights and access rules. LBRY also provides a data network that consists of peers (seeders) uploading and downloading data from other peers, possibly in exchange for payments, as well as a distributed hash table used by peers to discover other peers.
LBRY is an open-source protocol providing distribution, discovery, and purchase of digital content (data) via a decentralized network.
LBRY SDK for Python is currently the most full featured implementation of the LBRY Network protocols and includes many useful components and tools for building decentralized applications. Primary features and components:
This repo is a reference implementation of the [LBRY API](https://lbry.io/api).
* Built on Python 3.7+ and `asyncio`.
* DHT (Distributed Hash Table) implementation for finding peers ([lbrynet.dht](https://github.com/lbryio/lbry/tree/master/lbrynet/dht)).
* Blob exchange protocol for downloading content and negotiating payments ([lbrynet.blob_exchange](https://github.com/lbryio/lbry/tree/master/lbrynet/blob_exchange)).
* Protobuf schema for encoding and decoding metadata stored on the blockchain ([lbrynet.schema](https://github.com/lbryio/lbry/tree/master/lbrynet/schema)).
* Wallet implementation for the LBRY blockchain ([lbrynet.extras.wallet](https://github.com/lbryio/lbry/tree/master/lbrynet/extras/wallet)).
* Daemon with a JSON-RPC API to ease building end user applications in any language and for automating various tasks ([lbrynet.extras.daemon](https://github.com/lbryio/lbry/tree/master/lbrynet/extras/daemon)).
It provides a daemon that can interact with the network via a json-rpc interface over HTTP.
## Installation
## Installing
Our [releases page](https://github.com/lbryio/lbry/releases) contains pre-built binaries of the latest release, pre-releases, and past releases for macOS, Debian-based Linux, and Windows. [Automated travis builds](http://build.lbry.io/daemon/) are also available for testing.
**Note**: This project no longer bundles a graphic interface (browser). If you want to use LBRY via a GUI, [use the LBRY App](https://github.com/lbryio/lbry-app).
## Usage
Our [releases page](https://github.com/lbryio/lbry/releases) contains pre-built binaries of the latest release, pre-releases, and past releases, for macOS, Debian-based Linux, and Windows.
Run `lbrynet start` to launch the API server.
Installing from source is also relatively painless, full instructions are in [INSTALL.md](INSTALL.md)
By default, `lbrynet` will provide a JSON-RPC server at `http://localhost:5279`. It is easy to interact with via cURL or sane programming languages.
## Running
Our [quickstart guide](https://lbry.tech/playground) provides a simple walkthrough and examples for learning.
Run `lbrynet-daemon` to launch the daemon.
With the daemon running, `lbrynet commands` will show you a list of commands.
## Using
The full API is documented [here](https://lbry.tech/api/sdk).
By default, `lbrynet-daemon` will provide a JSON-RPC server at `http://localhost:5279`. It is easy to interact with via cURL or sane programming languages.
## Running from source
Our [quickstart guide](http://lbry.io/quickstart) provides a simple walkthrough and examples for learning.
Installing from source is also relatively painless. Full instructions are in [INSTALL.md](INSTALL.md)
The full API is documented [here](https://lbryio.github.io/lbry/cli).
## Contributing
## What is LBRY?
LBRY is a protocol that provides a fully decentralized network for the discovery, distribution, and payment of data.
It utilizes the [LBRY blockchain](https://github.com/lbryio/lbrycrd) as a global namespace and database of digital content. Blockchain entries contain searchable content metadata, identities, and rights and access rules.
LBRY also provides a data network consists of peers uploading and downloading data from other peers, possibly in exchange for payments, and a distributed hash table, used by peers to discover other peers.
## Contributions
To contribute, [join us on Discord](https://chat.lbry.io/) or contact jeremy@lbry.io. Pull requests are also welcome.
## Support
Please open an issue and describe your situation in detail. We will respond as soon as we can.
For private issues, contact jeremy@lbry.io.
Contributions to this project are welcome, encouraged, and compensated. For more details, please check [this](https://lbry.io/faq/contributing) link.
## License
See [LICENSE](LICENSE)
This project is MIT licensed. For the full license, see [LICENSE](LICENSE).
## Security
We take security seriously. Please contact security@lbry.io regarding any security issues. [Our GPG key is here](https://lbry.io/faq/gpg-key) if you need it.
## Contact
The primary contact for this project is [@eukreign](mailto:lex@lbry.io).
## Additional information and links
The documentation for the API can be found [here](https://lbry.tech/api/sdk).
Daemon defaults, ports, and other settings are documented [here](https://lbry.tech/resources/daemon-settings).
Settings can be configured using a daemon-settings.yml file. An example can be found [here](https://github.com/lbryio/lbry/blob/master/example_daemon_settings.yml).
+5
View File
@@ -0,0 +1,5 @@
build:
python setup.py apk
adb install -r lbry-0.1-service-debug.apk
.PHONY: build
+49
View File
@@ -0,0 +1,49 @@
import inspect
from os.path import dirname, join
import sh
from pythonforandroid.util import ensure_dir
from pythonforandroid.toolchain import (
Bootstrap, current_directory, info, info_main, shprint
)
class LBRYServiceBootstrap(Bootstrap):
name = 'lbry-service'
recipe_depends = ['genericndkbuild', 'python3']
bootstrap_dir = dirname(__file__)
def get_common_dir(self):
return join(dirname(inspect.getfile(Bootstrap)), 'bootstraps', 'common')
def run_distribute(self):
info_main('# Creating Android project from build and {} bootstrap'.format(
self.name))
info('This currently just copies the build stuff straight from the build dir.')
shprint(sh.rm, '-rf', self.dist_dir)
shprint(sh.cp, '-r', self.build_dir, self.dist_dir)
with current_directory(self.dist_dir):
with open('local.properties', 'w') as fileh:
fileh.write('sdk.dir={}'.format(self.ctx.sdk_dir))
arch = self.ctx.archs[0]
if len(self.ctx.archs) > 1:
raise ValueError('built for more than one arch, but bootstrap cannot handle that yet')
info('Bootstrap running with arch {}'.format(arch))
with current_directory(self.dist_dir):
info('Copying python distribution')
self.distribute_libs(arch, [self.ctx.get_libs_dir(arch.arch)])
self.distribute_aars(arch)
self.distribute_javaclasses(self.ctx.javaclass_dir)
python_bundle_dir = join('_python_bundle', '_python_bundle')
ensure_dir(python_bundle_dir)
site_packages_dir = self.ctx.python_recipe.create_python_bundle(
join(self.dist_dir, python_bundle_dir), arch)
self.strip_libraries(arch)
self.fry_eggs(site_packages_dir)
super().run_distribute()
+89
View File
@@ -0,0 +1,89 @@
# prevent user to include invalid extensions
*.apk
*.pxd
# eggs
*.egg-info
# unit test
unittest/*
# python config
config/makesetup
# unused kivy files (platform specific)
kivy/input/providers/wm_*
kivy/input/providers/mactouch*
kivy/input/providers/probesysfs*
kivy/input/providers/mtdev*
kivy/input/providers/hidinput*
kivy/core/camera/camera_videocapture*
kivy/core/spelling/*osx*
kivy/core/video/video_pyglet*
kivy/tools
kivy/tests/*
kivy/*/*.h
kivy/*/*.pxi
# unused encodings
lib-dynload/*codec*
encodings/cp*.pyo
encodings/tis*
encodings/shift*
encodings/bz2*
encodings/iso*
encodings/undefined*
encodings/johab*
encodings/p*
encodings/m*
encodings/euc*
encodings/k*
encodings/unicode_internal*
encodings/quo*
encodings/gb*
encodings/big5*
encodings/hp*
encodings/hz*
# unused python modules
bsddb/*
wsgiref/*
hotshot/*
pydoc_data/*
tty.pyo
anydbm.pyo
nturl2path.pyo
LICENCE.txt
macurl2path.pyo
dummy_threading.pyo
audiodev.pyo
antigravity.pyo
dumbdbm.pyo
sndhdr.pyo
__phello__.foo.pyo
sunaudio.pyo
os2emxpath.pyo
multiprocessing/dummy*
# unused binaries python modules
lib-dynload/termios.so
lib-dynload/_lsprof.so
lib-dynload/*audioop.so
lib-dynload/_hotshot.so
lib-dynload/_heapq.so
lib-dynload/_json.so
lib-dynload/grp.so
lib-dynload/resource.so
lib-dynload/pyexpat.so
lib-dynload/_ctypes_test.so
lib-dynload/_testcapi.so
# odd files
plat-linux3/regen
#>sqlite3
# conditionnal include depending if some recipes are included or not.
sqlite3/*
lib-dynload/_sqlite3.so
#<sqlite3
+1
View File
@@ -0,0 +1 @@
include $(call all-subdir-makefiles)
@@ -0,0 +1,7 @@
# Uncomment this if you're using STL in your project
# See CPLUSPLUS-SUPPORT.html in the NDK documentation for more information
# APP_STL := stlport_static
# APP_ABI := armeabi armeabi-v7a x86
APP_ABI := $(ARCH)
@@ -0,0 +1,22 @@
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := main
# Add your application source files here...
LOCAL_SRC_FILES := start.c pyjniusjni.c
LOCAL_CFLAGS += -I$(LOCAL_PATH)/../../../../../other_builds/$(MK_PYTHON_INCLUDE_ROOT) $(EXTRA_CFLAGS)
LOCAL_SHARED_LIBRARIES := python_shared
LOCAL_LDLIBS := -llog $(EXTRA_LDLIBS)
LOCAL_LDFLAGS += -L$(LOCAL_PATH)/../../../../../other_builds/$(MK_PYTHON_LINK_ROOT) $(APPLICATION_ADDITIONAL_LDFLAGS)
include $(BUILD_SHARED_LIBRARY)
ifdef CRYSTAX_PYTHON_VERSION
$(call import-module,python/$(CRYSTAX_PYTHON_VERSION))
endif
@@ -0,0 +1,10 @@
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := main
LOCAL_SRC_FILES := YourSourceHere.c
include $(BUILD_SHARED_LIBRARY)
$(call import-module,SDL)LOCAL_PATH := $(call my-dir)
@@ -0,0 +1,6 @@
#define BOOTSTRAP_NAME_SERVICEONLY
#define BOOTSTRAP_USES_NO_SDL_HEADERS
const char bootstrap_name[] = "service_only";
@@ -0,0 +1,103 @@
#include <pthread.h>
#include <jni.h>
#define LOGI(...) do {} while (0)
#define LOGE(...) do {} while (0)
#include "android/log.h"
/* These JNI management functions are taken from SDL2, but modified to refer to pyjnius */
/* #define LOG(n, x) __android_log_write(ANDROID_LOG_INFO, (n), (x)) */
/* #define LOGP(x) LOG("python", (x)) */
#define LOG_TAG "Python_android"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO,LOG_TAG,__VA_ARGS__)
#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR,LOG_TAG,__VA_ARGS__)
/* Function headers */
JNIEnv* Android_JNI_GetEnv(void);
static void Android_JNI_ThreadDestroyed(void*);
static pthread_key_t mThreadKey;
static JavaVM* mJavaVM;
int Android_JNI_SetupThread(void)
{
Android_JNI_GetEnv();
return 1;
}
/* Library init */
JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved)
{
JNIEnv *env;
mJavaVM = vm;
LOGI("JNI_OnLoad called");
if ((*mJavaVM)->GetEnv(mJavaVM, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
LOGE("Failed to get the environment using GetEnv()");
return -1;
}
/*
* Create mThreadKey so we can keep track of the JNIEnv assigned to each thread
* Refer to http://developer.android.com/guide/practices/design/jni.html for the rationale behind this
*/
if (pthread_key_create(&mThreadKey, Android_JNI_ThreadDestroyed) != 0) {
__android_log_print(ANDROID_LOG_ERROR, "pyjniusjni", "Error initializing pthread key");
}
Android_JNI_SetupThread();
return JNI_VERSION_1_4;
}
JNIEnv* Android_JNI_GetEnv(void)
{
/* From http://developer.android.com/guide/practices/jni.html
* All threads are Linux threads, scheduled by the kernel.
* They're usually started from managed code (using Thread.start), but they can also be created elsewhere and then
* attached to the JavaVM. For example, a thread started with pthread_create can be attached with the
* JNI AttachCurrentThread or AttachCurrentThreadAsDaemon functions. Until a thread is attached, it has no JNIEnv,
* and cannot make JNI calls.
* Attaching a natively-created thread causes a java.lang.Thread object to be constructed and added to the "main"
* ThreadGroup, making it visible to the debugger. Calling AttachCurrentThread on an already-attached thread
* is a no-op.
* Note: You can call this function any number of times for the same thread, there's no harm in it
*/
JNIEnv *env;
int status = (*mJavaVM)->AttachCurrentThread(mJavaVM, &env, NULL);
if(status < 0) {
LOGE("failed to attach current thread");
return 0;
}
/* From http://developer.android.com/guide/practices/jni.html
* Threads attached through JNI must call DetachCurrentThread before they exit. If coding this directly is awkward,
* in Android 2.0 (Eclair) and higher you can use pthread_key_create to define a destructor function that will be
* called before the thread exits, and call DetachCurrentThread from there. (Use that key with pthread_setspecific
* to store the JNIEnv in thread-local-storage; that way it'll be passed into your destructor as the argument.)
* Note: The destructor is not called unless the stored value is != NULL
* Note: You can call this function any number of times for the same thread, there's no harm in it
* (except for some lost CPU cycles)
*/
pthread_setspecific(mThreadKey, (void*) env);
return env;
}
static void Android_JNI_ThreadDestroyed(void* value)
{
/* The thread is being destroyed, detach it from the Java VM and set the mThreadKey value to NULL as required */
JNIEnv *env = (JNIEnv*) value;
if (env != NULL) {
(*mJavaVM)->DetachCurrentThread(mJavaVM);
pthread_setspecific(mThreadKey, NULL);
}
}
void *WebView_AndroidGetJNIEnv()
{
return Android_JNI_GetEnv();
}
@@ -0,0 +1,141 @@
/**
* Copyright 2012 Kamran Zafar
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.kamranzafar.jtar;
/**
* @author Kamran Zafar
*
*/
public class Octal {
/**
* Parse an octal string from a header buffer. This is used for the file
* permission mode value.
*
* @param header
* The header buffer from which to parse.
* @param offset
* The offset into the buffer from which to parse.
* @param length
* The number of header bytes to parse.
*
* @return The long value of the octal string.
*/
public static long parseOctal(byte[] header, int offset, int length) {
long result = 0;
boolean stillPadding = true;
int end = offset + length;
for (int i = offset; i < end; ++i) {
if (header[i] == 0)
break;
if (header[i] == (byte) ' ' || header[i] == '0') {
if (stillPadding)
continue;
if (header[i] == (byte) ' ')
break;
}
stillPadding = false;
result = ( result << 3 ) + ( header[i] - '0' );
}
return result;
}
/**
* Parse an octal integer from a header buffer.
*
* @param value
* @param buf
* The header buffer from which to parse.
* @param offset
* The offset into the buffer from which to parse.
* @param length
* The number of header bytes to parse.
*
* @return The integer value of the octal bytes.
*/
public static int getOctalBytes(long value, byte[] buf, int offset, int length) {
int idx = length - 1;
buf[offset + idx] = 0;
--idx;
buf[offset + idx] = (byte) ' ';
--idx;
if (value == 0) {
buf[offset + idx] = (byte) '0';
--idx;
} else {
for (long val = value; idx >= 0 && val > 0; --idx) {
buf[offset + idx] = (byte) ( (byte) '0' + (byte) ( val & 7 ) );
val = val >> 3;
}
}
for (; idx >= 0; --idx) {
buf[offset + idx] = (byte) ' ';
}
return offset + length;
}
/**
* Parse the checksum octal integer from a header buffer.
*
* @param value
* @param buf
* The header buffer from which to parse.
* @param offset
* The offset into the buffer from which to parse.
* @param length
* The number of header bytes to parse.
* @return The integer value of the entry's checksum.
*/
public static int getCheckSumOctalBytes(long value, byte[] buf, int offset, int length) {
getOctalBytes( value, buf, offset, length );
buf[offset + length - 1] = (byte) ' ';
buf[offset + length - 2] = 0;
return offset + length;
}
/**
* Parse an octal long integer from a header buffer.
*
* @param value
* @param buf
* The header buffer from which to parse.
* @param offset
* The offset into the buffer from which to parse.
* @param length
* The number of header bytes to parse.
*
* @return The long value of the octal bytes.
*/
public static int getLongOctalBytes(long value, byte[] buf, int offset, int length) {
byte[] temp = new byte[length + 1];
getOctalBytes( value, temp, 0, length + 1 );
System.arraycopy( temp, 0, buf, offset, length );
return offset + length;
}
}
@@ -0,0 +1,28 @@
/**
* Copyright 2012 Kamran Zafar
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.kamranzafar.jtar;
/**
* @author Kamran Zafar
*
*/
public class TarConstants {
public static final int EOF_BLOCK = 1024;
public static final int DATA_BLOCK = 512;
public static final int HEADER_BLOCK = 512;
}
@@ -0,0 +1,284 @@
/**
* Copyright 2012 Kamran Zafar
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.kamranzafar.jtar;
import java.io.File;
import java.util.Date;
/**
* @author Kamran Zafar
*
*/
public class TarEntry {
protected File file;
protected TarHeader header;
private TarEntry() {
this.file = null;
header = new TarHeader();
}
public TarEntry(File file, String entryName) {
this();
this.file = file;
this.extractTarHeader(entryName);
}
public TarEntry(byte[] headerBuf) {
this();
this.parseTarHeader(headerBuf);
}
/**
* Constructor to create an entry from an existing TarHeader object.
*
* This method is useful to add new entries programmatically (e.g. for
* adding files or directories that do not exist in the file system).
*
* @param header
*
*/
public TarEntry(TarHeader header) {
this.file = null;
this.header = header;
}
public boolean equals(TarEntry it) {
return header.name.toString().equals(it.header.name.toString());
}
public boolean isDescendent(TarEntry desc) {
return desc.header.name.toString().startsWith(header.name.toString());
}
public TarHeader getHeader() {
return header;
}
public String getName() {
String name = header.name.toString();
if (header.namePrefix != null && !header.namePrefix.toString().equals("")) {
name = header.namePrefix.toString() + "/" + name;
}
return name;
}
public void setName(String name) {
header.name = new StringBuffer(name);
}
public int getUserId() {
return header.userId;
}
public void setUserId(int userId) {
header.userId = userId;
}
public int getGroupId() {
return header.groupId;
}
public void setGroupId(int groupId) {
header.groupId = groupId;
}
public String getUserName() {
return header.userName.toString();
}
public void setUserName(String userName) {
header.userName = new StringBuffer(userName);
}
public String getGroupName() {
return header.groupName.toString();
}
public void setGroupName(String groupName) {
header.groupName = new StringBuffer(groupName);
}
public void setIds(int userId, int groupId) {
this.setUserId(userId);
this.setGroupId(groupId);
}
public void setModTime(long time) {
header.modTime = time / 1000;
}
public void setModTime(Date time) {
header.modTime = time.getTime() / 1000;
}
public Date getModTime() {
return new Date(header.modTime * 1000);
}
public File getFile() {
return this.file;
}
public long getSize() {
return header.size;
}
public void setSize(long size) {
header.size = size;
}
/**
* Checks if the org.kamrazafar.jtar entry is a directory
*
* @return
*/
public boolean isDirectory() {
if (this.file != null)
return this.file.isDirectory();
if (header != null) {
if (header.linkFlag == TarHeader.LF_DIR)
return true;
if (header.name.toString().endsWith("/"))
return true;
}
return false;
}
/**
* Extract header from File
*
* @param entryName
*/
public void extractTarHeader(String entryName) {
header = TarHeader.createHeader(entryName, file.length(), file.lastModified() / 1000, file.isDirectory());
}
/**
* Calculate checksum
*
* @param buf
* @return
*/
public long computeCheckSum(byte[] buf) {
long sum = 0;
for (int i = 0; i < buf.length; ++i) {
sum += 255 & buf[i];
}
return sum;
}
/**
* Writes the header to the byte buffer
*
* @param outbuf
*/
public void writeEntryHeader(byte[] outbuf) {
int offset = 0;
offset = TarHeader.getNameBytes(header.name, outbuf, offset, TarHeader.NAMELEN);
offset = Octal.getOctalBytes(header.mode, outbuf, offset, TarHeader.MODELEN);
offset = Octal.getOctalBytes(header.userId, outbuf, offset, TarHeader.UIDLEN);
offset = Octal.getOctalBytes(header.groupId, outbuf, offset, TarHeader.GIDLEN);
long size = header.size;
offset = Octal.getLongOctalBytes(size, outbuf, offset, TarHeader.SIZELEN);
offset = Octal.getLongOctalBytes(header.modTime, outbuf, offset, TarHeader.MODTIMELEN);
int csOffset = offset;
for (int c = 0; c < TarHeader.CHKSUMLEN; ++c)
outbuf[offset++] = (byte) ' ';
outbuf[offset++] = header.linkFlag;
offset = TarHeader.getNameBytes(header.linkName, outbuf, offset, TarHeader.NAMELEN);
offset = TarHeader.getNameBytes(header.magic, outbuf, offset, TarHeader.USTAR_MAGICLEN);
offset = TarHeader.getNameBytes(header.userName, outbuf, offset, TarHeader.USTAR_USER_NAMELEN);
offset = TarHeader.getNameBytes(header.groupName, outbuf, offset, TarHeader.USTAR_GROUP_NAMELEN);
offset = Octal.getOctalBytes(header.devMajor, outbuf, offset, TarHeader.USTAR_DEVLEN);
offset = Octal.getOctalBytes(header.devMinor, outbuf, offset, TarHeader.USTAR_DEVLEN);
offset = TarHeader.getNameBytes(header.namePrefix, outbuf, offset, TarHeader.USTAR_FILENAME_PREFIX);
for (; offset < outbuf.length;)
outbuf[offset++] = 0;
long checkSum = this.computeCheckSum(outbuf);
Octal.getCheckSumOctalBytes(checkSum, outbuf, csOffset, TarHeader.CHKSUMLEN);
}
/**
* Parses the tar header to the byte buffer
*
* @param header
* @param bh
*/
public void parseTarHeader(byte[] bh) {
int offset = 0;
header.name = TarHeader.parseName(bh, offset, TarHeader.NAMELEN);
offset += TarHeader.NAMELEN;
header.mode = (int) Octal.parseOctal(bh, offset, TarHeader.MODELEN);
offset += TarHeader.MODELEN;
header.userId = (int) Octal.parseOctal(bh, offset, TarHeader.UIDLEN);
offset += TarHeader.UIDLEN;
header.groupId = (int) Octal.parseOctal(bh, offset, TarHeader.GIDLEN);
offset += TarHeader.GIDLEN;
header.size = Octal.parseOctal(bh, offset, TarHeader.SIZELEN);
offset += TarHeader.SIZELEN;
header.modTime = Octal.parseOctal(bh, offset, TarHeader.MODTIMELEN);
offset += TarHeader.MODTIMELEN;
header.checkSum = (int) Octal.parseOctal(bh, offset, TarHeader.CHKSUMLEN);
offset += TarHeader.CHKSUMLEN;
header.linkFlag = bh[offset++];
header.linkName = TarHeader.parseName(bh, offset, TarHeader.NAMELEN);
offset += TarHeader.NAMELEN;
header.magic = TarHeader.parseName(bh, offset, TarHeader.USTAR_MAGICLEN);
offset += TarHeader.USTAR_MAGICLEN;
header.userName = TarHeader.parseName(bh, offset, TarHeader.USTAR_USER_NAMELEN);
offset += TarHeader.USTAR_USER_NAMELEN;
header.groupName = TarHeader.parseName(bh, offset, TarHeader.USTAR_GROUP_NAMELEN);
offset += TarHeader.USTAR_GROUP_NAMELEN;
header.devMajor = (int) Octal.parseOctal(bh, offset, TarHeader.USTAR_DEVLEN);
offset += TarHeader.USTAR_DEVLEN;
header.devMinor = (int) Octal.parseOctal(bh, offset, TarHeader.USTAR_DEVLEN);
offset += TarHeader.USTAR_DEVLEN;
header.namePrefix = TarHeader.parseName(bh, offset, TarHeader.USTAR_FILENAME_PREFIX);
}
}
@@ -0,0 +1,243 @@
/**
* Copyright 2012 Kamran Zafar
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.kamranzafar.jtar;
import java.io.File;
/**
* Header
*
* <pre>
* Offset Size Field
* 0 100 File name
* 100 8 File mode
* 108 8 Owner's numeric user ID
* 116 8 Group's numeric user ID
* 124 12 File size in bytes
* 136 12 Last modification time in numeric Unix time format
* 148 8 Checksum for header block
* 156 1 Link indicator (file type)
* 157 100 Name of linked file
* </pre>
*
*
* File Types
*
* <pre>
* Value Meaning
* '0' Normal file
* (ASCII NUL) Normal file (now obsolete)
* '1' Hard link
* '2' Symbolic link
* '3' Character special
* '4' Block special
* '5' Directory
* '6' FIFO
* '7' Contigous
* </pre>
*
*
*
* Ustar header
*
* <pre>
* Offset Size Field
* 257 6 UStar indicator "ustar"
* 263 2 UStar version "00"
* 265 32 Owner user name
* 297 32 Owner group name
* 329 8 Device major number
* 337 8 Device minor number
* 345 155 Filename prefix
* </pre>
*/
public class TarHeader {
/*
* Header
*/
public static final int NAMELEN = 100;
public static final int MODELEN = 8;
public static final int UIDLEN = 8;
public static final int GIDLEN = 8;
public static final int SIZELEN = 12;
public static final int MODTIMELEN = 12;
public static final int CHKSUMLEN = 8;
public static final byte LF_OLDNORM = 0;
/*
* File Types
*/
public static final byte LF_NORMAL = (byte) '0';
public static final byte LF_LINK = (byte) '1';
public static final byte LF_SYMLINK = (byte) '2';
public static final byte LF_CHR = (byte) '3';
public static final byte LF_BLK = (byte) '4';
public static final byte LF_DIR = (byte) '5';
public static final byte LF_FIFO = (byte) '6';
public static final byte LF_CONTIG = (byte) '7';
/*
* Ustar header
*/
public static final String USTAR_MAGIC = "ustar"; // POSIX
public static final int USTAR_MAGICLEN = 8;
public static final int USTAR_USER_NAMELEN = 32;
public static final int USTAR_GROUP_NAMELEN = 32;
public static final int USTAR_DEVLEN = 8;
public static final int USTAR_FILENAME_PREFIX = 155;
// Header values
public StringBuffer name;
public int mode;
public int userId;
public int groupId;
public long size;
public long modTime;
public int checkSum;
public byte linkFlag;
public StringBuffer linkName;
public StringBuffer magic; // ustar indicator and version
public StringBuffer userName;
public StringBuffer groupName;
public int devMajor;
public int devMinor;
public StringBuffer namePrefix;
public TarHeader() {
this.magic = new StringBuffer(TarHeader.USTAR_MAGIC);
this.name = new StringBuffer();
this.linkName = new StringBuffer();
String user = System.getProperty("user.name", "");
if (user.length() > 31)
user = user.substring(0, 31);
this.userId = 0;
this.groupId = 0;
this.userName = new StringBuffer(user);
this.groupName = new StringBuffer("");
this.namePrefix = new StringBuffer();
}
/**
* Parse an entry name from a header buffer.
*
* @param name
* @param header
* The header buffer from which to parse.
* @param offset
* The offset into the buffer from which to parse.
* @param length
* The number of header bytes to parse.
* @return The header's entry name.
*/
public static StringBuffer parseName(byte[] header, int offset, int length) {
StringBuffer result = new StringBuffer(length);
int end = offset + length;
for (int i = offset; i < end; ++i) {
if (header[i] == 0)
break;
result.append((char) header[i]);
}
return result;
}
/**
* Determine the number of bytes in an entry name.
*
* @param name
* @param header
* The header buffer from which to parse.
* @param offset
* The offset into the buffer from which to parse.
* @param length
* The number of header bytes to parse.
* @return The number of bytes in a header's entry name.
*/
public static int getNameBytes(StringBuffer name, byte[] buf, int offset, int length) {
int i;
for (i = 0; i < length && i < name.length(); ++i) {
buf[offset + i] = (byte) name.charAt(i);
}
for (; i < length; ++i) {
buf[offset + i] = 0;
}
return offset + length;
}
/**
* Creates a new header for a file/directory entry.
*
*
* @param name
* File name
* @param size
* File size in bytes
* @param modTime
* Last modification time in numeric Unix time format
* @param dir
* Is directory
*
* @return
*/
public static TarHeader createHeader(String entryName, long size, long modTime, boolean dir) {
String name = entryName;
name = TarUtils.trim(name.replace(File.separatorChar, '/'), '/');
TarHeader header = new TarHeader();
header.linkName = new StringBuffer("");
if (name.length() > 100) {
header.namePrefix = new StringBuffer(name.substring(0, name.lastIndexOf('/')));
header.name = new StringBuffer(name.substring(name.lastIndexOf('/') + 1));
} else {
header.name = new StringBuffer(name);
}
if (dir) {
header.mode = 040755;
header.linkFlag = TarHeader.LF_DIR;
if (header.name.charAt(header.name.length() - 1) != '/') {
header.name.append("/");
}
header.size = 0;
} else {
header.mode = 0100644;
header.linkFlag = TarHeader.LF_NORMAL;
header.size = size;
}
header.modTime = modTime;
header.checkSum = 0;
header.devMajor = 0;
header.devMinor = 0;
return header;
}
}
@@ -0,0 +1,249 @@
/**
* Copyright 2012 Kamran Zafar
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.kamranzafar.jtar;
import java.io.FilterInputStream;
import java.io.IOException;
import java.io.InputStream;
/**
* @author Kamran Zafar
*
*/
public class TarInputStream extends FilterInputStream {
private static final int SKIP_BUFFER_SIZE = 2048;
private TarEntry currentEntry;
private long currentFileSize;
private long bytesRead;
private boolean defaultSkip = false;
public TarInputStream(InputStream in) {
super(in);
currentFileSize = 0;
bytesRead = 0;
}
@Override
public boolean markSupported() {
return false;
}
/**
* Not supported
*
*/
@Override
public synchronized void mark(int readlimit) {
}
/**
* Not supported
*
*/
@Override
public synchronized void reset() throws IOException {
throw new IOException("mark/reset not supported");
}
/**
* Read a byte
*
* @see java.io.FilterInputStream#read()
*/
@Override
public int read() throws IOException {
byte[] buf = new byte[1];
int res = this.read(buf, 0, 1);
if (res != -1) {
return 0xFF & buf[0];
}
return res;
}
/**
* Checks if the bytes being read exceed the entry size and adjusts the byte
* array length. Updates the byte counters
*
*
* @see java.io.FilterInputStream#read(byte[], int, int)
*/
@Override
public int read(byte[] b, int off, int len) throws IOException {
if (currentEntry != null) {
if (currentFileSize == currentEntry.getSize()) {
return -1;
} else if ((currentEntry.getSize() - currentFileSize) < len) {
len = (int) (currentEntry.getSize() - currentFileSize);
}
}
int br = super.read(b, off, len);
if (br != -1) {
if (currentEntry != null) {
currentFileSize += br;
}
bytesRead += br;
}
return br;
}
/**
* Returns the next entry in the tar file
*
* @return TarEntry
* @throws IOException
*/
public TarEntry getNextEntry() throws IOException {
closeCurrentEntry();
byte[] header = new byte[TarConstants.HEADER_BLOCK];
byte[] theader = new byte[TarConstants.HEADER_BLOCK];
int tr = 0;
// Read full header
while (tr < TarConstants.HEADER_BLOCK) {
int res = read(theader, 0, TarConstants.HEADER_BLOCK - tr);
if (res < 0) {
break;
}
System.arraycopy(theader, 0, header, tr, res);
tr += res;
}
// Check if record is null
boolean eof = true;
for (byte b : header) {
if (b != 0) {
eof = false;
break;
}
}
if (!eof) {
currentEntry = new TarEntry(header);
}
return currentEntry;
}
/**
* Returns the current offset (in bytes) from the beginning of the stream.
* This can be used to find out at which point in a tar file an entry's content begins, for instance.
*/
public long getCurrentOffset() {
return bytesRead;
}
/**
* Closes the current tar entry
*
* @throws IOException
*/
protected void closeCurrentEntry() throws IOException {
if (currentEntry != null) {
if (currentEntry.getSize() > currentFileSize) {
// Not fully read, skip rest of the bytes
long bs = 0;
while (bs < currentEntry.getSize() - currentFileSize) {
long res = skip(currentEntry.getSize() - currentFileSize - bs);
if (res == 0 && currentEntry.getSize() - currentFileSize > 0) {
// I suspect file corruption
throw new IOException("Possible tar file corruption");
}
bs += res;
}
}
currentEntry = null;
currentFileSize = 0L;
skipPad();
}
}
/**
* Skips the pad at the end of each tar entry file content
*
* @throws IOException
*/
protected void skipPad() throws IOException {
if (bytesRead > 0) {
int extra = (int) (bytesRead % TarConstants.DATA_BLOCK);
if (extra > 0) {
long bs = 0;
while (bs < TarConstants.DATA_BLOCK - extra) {
long res = skip(TarConstants.DATA_BLOCK - extra - bs);
bs += res;
}
}
}
}
/**
* Skips 'n' bytes on the InputStream<br>
* Overrides default implementation of skip
*
*/
@Override
public long skip(long n) throws IOException {
if (defaultSkip) {
// use skip method of parent stream
// may not work if skip not implemented by parent
long bs = super.skip(n);
bytesRead += bs;
return bs;
}
if (n <= 0) {
return 0;
}
long left = n;
byte[] sBuff = new byte[SKIP_BUFFER_SIZE];
while (left > 0) {
int res = read(sBuff, 0, (int) (left < SKIP_BUFFER_SIZE ? left : SKIP_BUFFER_SIZE));
if (res < 0) {
break;
}
left -= res;
}
return n - left;
}
public boolean isDefaultSkip() {
return defaultSkip;
}
public void setDefaultSkip(boolean defaultSkip) {
this.defaultSkip = defaultSkip;
}
}
@@ -0,0 +1,163 @@
/**
* Copyright 2012 Kamran Zafar
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.kamranzafar.jtar;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.RandomAccessFile;
/**
* @author Kamran Zafar
*
*/
public class TarOutputStream extends OutputStream {
private final OutputStream out;
private long bytesWritten;
private long currentFileSize;
private TarEntry currentEntry;
public TarOutputStream(OutputStream out) {
this.out = out;
bytesWritten = 0;
currentFileSize = 0;
}
public TarOutputStream(final File fout) throws FileNotFoundException {
this.out = new BufferedOutputStream(new FileOutputStream(fout));
bytesWritten = 0;
currentFileSize = 0;
}
/**
* Opens a file for writing.
*/
public TarOutputStream(final File fout, final boolean append) throws IOException {
@SuppressWarnings("resource")
RandomAccessFile raf = new RandomAccessFile(fout, "rw");
final long fileSize = fout.length();
if (append && fileSize > TarConstants.EOF_BLOCK) {
raf.seek(fileSize - TarConstants.EOF_BLOCK);
}
out = new BufferedOutputStream(new FileOutputStream(raf.getFD()));
}
/**
* Appends the EOF record and closes the stream
*
* @see java.io.FilterOutputStream#close()
*/
@Override
public void close() throws IOException {
closeCurrentEntry();
write( new byte[TarConstants.EOF_BLOCK] );
out.close();
}
/**
* Writes a byte to the stream and updates byte counters
*
* @see java.io.FilterOutputStream#write(int)
*/
@Override
public void write(int b) throws IOException {
out.write( b );
bytesWritten += 1;
if (currentEntry != null) {
currentFileSize += 1;
}
}
/**
* Checks if the bytes being written exceed the current entry size.
*
* @see java.io.FilterOutputStream#write(byte[], int, int)
*/
@Override
public void write(byte[] b, int off, int len) throws IOException {
if (currentEntry != null && !currentEntry.isDirectory()) {
if (currentEntry.getSize() < currentFileSize + len) {
throw new IOException( "The current entry[" + currentEntry.getName() + "] size["
+ currentEntry.getSize() + "] is smaller than the bytes[" + ( currentFileSize + len )
+ "] being written." );
}
}
out.write( b, off, len );
bytesWritten += len;
if (currentEntry != null) {
currentFileSize += len;
}
}
/**
* Writes the next tar entry header on the stream
*
* @param entry
* @throws IOException
*/
public void putNextEntry(TarEntry entry) throws IOException {
closeCurrentEntry();
byte[] header = new byte[TarConstants.HEADER_BLOCK];
entry.writeEntryHeader( header );
write( header );
currentEntry = entry;
}
/**
* Closes the current tar entry
*
* @throws IOException
*/
protected void closeCurrentEntry() throws IOException {
if (currentEntry != null) {
if (currentEntry.getSize() > currentFileSize) {
throw new IOException( "The current entry[" + currentEntry.getName() + "] of size["
+ currentEntry.getSize() + "] has not been fully written." );
}
currentEntry = null;
currentFileSize = 0;
pad();
}
}
/**
* Pads the last content block
*
* @throws IOException
*/
protected void pad() throws IOException {
if (bytesWritten > 0) {
int extra = (int) ( bytesWritten % TarConstants.DATA_BLOCK );
if (extra > 0) {
write( new byte[TarConstants.DATA_BLOCK - extra] );
}
}
}
}
@@ -0,0 +1,96 @@
/**
* Copyright 2012 Kamran Zafar
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.kamranzafar.jtar;
import java.io.File;
/**
* @author Kamran
*
*/
public class TarUtils {
/**
* Determines the tar file size of the given folder/file path
*
* @param path
* @return
*/
public static long calculateTarSize(File path) {
return tarSize(path) + TarConstants.EOF_BLOCK;
}
private static long tarSize(File dir) {
long size = 0;
if (dir.isFile()) {
return entrySize(dir.length());
} else {
File[] subFiles = dir.listFiles();
if (subFiles != null && subFiles.length > 0) {
for (File file : subFiles) {
if (file.isFile()) {
size += entrySize(file.length());
} else {
size += tarSize(file);
}
}
} else {
// Empty folder header
return TarConstants.HEADER_BLOCK;
}
}
return size;
}
private static long entrySize(long fileSize) {
long size = 0;
size += TarConstants.HEADER_BLOCK; // Header
size += fileSize; // File size
long extra = size % TarConstants.DATA_BLOCK;
if (extra > 0) {
size += (TarConstants.DATA_BLOCK - extra); // pad
}
return size;
}
public static String trim(String s, char c) {
StringBuffer tmp = new StringBuffer(s);
for (int i = 0; i < tmp.length(); i++) {
if (tmp.charAt(i) != c) {
break;
} else {
tmp.deleteCharAt(i);
}
}
for (int i = tmp.length() - 1; i >= 0; i--) {
if (tmp.charAt(i) != c) {
break;
} else {
tmp.deleteCharAt(i);
}
}
return tmp.toString();
}
}
@@ -0,0 +1,400 @@
package org.kivy.android;
import java.net.Socket;
import java.net.InetSocketAddress;
import android.os.SystemClock;
import java.io.InputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.FileWriter;
import java.io.File;
import java.io.IOException;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.ArrayList;
import android.app.*;
import android.content.*;
import android.view.*;
import android.view.SurfaceView;
import android.app.Activity;
import android.content.Intent;
import android.util.Log;
import android.widget.Toast;
import android.os.AsyncTask;
import android.os.Bundle;
import android.os.PowerManager;
import android.graphics.PixelFormat;
import android.view.SurfaceHolder;
import android.content.Context;
import android.content.pm.ActivityInfo;
import android.content.pm.PackageManager;
import android.content.pm.ApplicationInfo;
import android.content.Intent;
import android.widget.ImageView;
import java.io.InputStream;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Color;
import android.widget.AbsoluteLayout;
import android.webkit.WebViewClient;
import android.webkit.WebView;
import org.kivy.android.PythonUtil;
import org.renpy.android.ResourceManager;
import org.renpy.android.AssetExtract;
public class PythonActivity extends Activity {
// This activity is modified from a mixture of the SDLActivity and
// PythonActivity in the SDL2 bootstrap, but removing all the SDL2
// specifics.
private static final String TAG = "PythonActivity";
public static PythonActivity mActivity = null;
/** If shared libraries (e.g. the native application) could not be loaded. */
public static boolean mBrokenLibraries;
protected static Thread mPythonThread;
private ResourceManager resourceManager = null;
private Bundle mMetaData = null;
private PowerManager.WakeLock mWakeLock = null;
public String getAppRoot() {
String app_root = getFilesDir().getAbsolutePath() + "/app";
return app_root;
}
public static void initialize() {
// The static nature of the singleton and Android quirkiness force us to initialize everything here
// Otherwise, when exiting the app and returning to it, these variables *keep* their pre exit values
mBrokenLibraries = false;
}
@Override
protected void onCreate(Bundle savedInstanceState) {
Log.v(TAG, "My oncreate running");
resourceManager = new ResourceManager(this);
Log.v(TAG, "Ready to unpack");
File app_root_file = new File(getAppRoot());
unpackData("private", app_root_file);
Log.v(TAG, "About to do super onCreate");
super.onCreate(savedInstanceState);
Log.v(TAG, "Did super onCreate");
this.mActivity = this;
//this.showLoadingScreen();
Log.v("Python", "Device: " + android.os.Build.DEVICE);
Log.v("Python", "Model: " + android.os.Build.MODEL);
//Log.v(TAG, "Ready to unpack");
//new UnpackFilesTask().execute(getAppRoot());
PythonActivity.initialize();
// Load shared libraries
String errorMsgBrokenLib = "";
try {
loadLibraries();
} catch(UnsatisfiedLinkError e) {
System.err.println(e.getMessage());
mBrokenLibraries = true;
errorMsgBrokenLib = e.getMessage();
} catch(Exception e) {
System.err.println(e.getMessage());
mBrokenLibraries = true;
errorMsgBrokenLib = e.getMessage();
}
if (mBrokenLibraries)
{
AlertDialog.Builder dlgAlert = new AlertDialog.Builder(this);
dlgAlert.setMessage("An error occurred while trying to load the application libraries. Please try again and/or reinstall."
+ System.getProperty("line.separator")
+ System.getProperty("line.separator")
+ "Error: " + errorMsgBrokenLib);
dlgAlert.setTitle("Python Error");
dlgAlert.setPositiveButton("Exit",
new DialogInterface.OnClickListener() {
@Override
public void onClick(DialogInterface dialog,int id) {
// if this button is clicked, close current activity
PythonActivity.mActivity.finish();
}
});
dlgAlert.setCancelable(false);
dlgAlert.create().show();
return;
}
// Set up the Python environment
String app_root_dir = getAppRoot();
String mFilesDirectory = mActivity.getFilesDir().getAbsolutePath();
Log.v(TAG, "Setting env vars for start.c and Python to use");
PythonActivity.nativeSetEnv("ANDROID_ENTRYPOINT", "main.pyo");
PythonActivity.nativeSetEnv("ANDROID_ARGUMENT", app_root_dir);
PythonActivity.nativeSetEnv("ANDROID_APP_PATH", app_root_dir);
PythonActivity.nativeSetEnv("ANDROID_PRIVATE", mFilesDirectory);
PythonActivity.nativeSetEnv("ANDROID_UNPACK", app_root_dir);
PythonActivity.nativeSetEnv("PYTHONHOME", app_root_dir);
PythonActivity.nativeSetEnv("PYTHONPATH", app_root_dir + ":" + app_root_dir + "/lib");
PythonActivity.nativeSetEnv("PYTHONOPTIMIZE", "2");
try {
Log.v(TAG, "Access to our meta-data...");
mActivity.mMetaData = mActivity.getPackageManager().getApplicationInfo(
mActivity.getPackageName(), PackageManager.GET_META_DATA).metaData;
PowerManager pm = (PowerManager) mActivity.getSystemService(Context.POWER_SERVICE);
if ( mActivity.mMetaData.getInt("wakelock") == 1 ) {
mActivity.mWakeLock = pm.newWakeLock(PowerManager.SCREEN_BRIGHT_WAKE_LOCK, "Screen On");
mActivity.mWakeLock.acquire();
}
} catch (PackageManager.NameNotFoundException e) {
}
final Thread pythonThread = new Thread(new PythonMain(), "PythonThread");
PythonActivity.mPythonThread = pythonThread;
pythonThread.start();
}
@Override
public void onDestroy() {
Log.i("Destroy", "end of app");
super.onDestroy();
// make sure all child threads (python_thread) are stopped
android.os.Process.killProcess(android.os.Process.myPid());
}
public void loadLibraries() {
String app_root = new String(getAppRoot());
File app_root_file = new File(app_root);
PythonUtil.loadLibraries(app_root_file);
}
public void recursiveDelete(File f) {
if (f.isDirectory()) {
for (File r : f.listFiles()) {
recursiveDelete(r);
}
}
f.delete();
}
/**
* Show an error using a toast. (Only makes sense from non-UI
* threads.)
*/
public void toastError(final String msg) {
final Activity thisActivity = this;
runOnUiThread(new Runnable () {
public void run() {
Toast.makeText(thisActivity, msg, Toast.LENGTH_LONG).show();
}
});
// Wait to show the error.
synchronized (this) {
try {
this.wait(1000);
} catch (InterruptedException e) {
}
}
}
public void unpackData(final String resource, File target) {
Log.v(TAG, "UNPACKING!!! " + resource + " " + target.getName());
// The version of data in memory and on disk.
String data_version = resourceManager.getString(resource + "_version");
String disk_version = null;
Log.v(TAG, "Data version is " + data_version);
// If no version, no unpacking is necessary.
if (data_version == null) {
return;
}
// Check the current disk version, if any.
String filesDir = target.getAbsolutePath();
String disk_version_fn = filesDir + "/" + resource + ".version";
try {
byte buf[] = new byte[64];
InputStream is = new FileInputStream(disk_version_fn);
int len = is.read(buf);
disk_version = new String(buf, 0, len);
is.close();
} catch (Exception e) {
disk_version = "";
}
// If the disk data is out of date, extract it and write the
// version file.
// if (! data_version.equals(disk_version)) {
if (! data_version.equals(disk_version)) {
Log.v(TAG, "Extracting " + resource + " assets.");
recursiveDelete(target);
target.mkdirs();
AssetExtract ae = new AssetExtract(this);
if (!ae.extractTar(resource + ".mp3", target.getAbsolutePath())) {
toastError("Could not extract " + resource + " data.");
}
try {
// Write .nomedia.
new File(target, ".nomedia").createNewFile();
// Write version file.
FileOutputStream os = new FileOutputStream(disk_version_fn);
os.write(data_version.getBytes());
os.close();
} catch (Exception e) {
Log.w("python", e);
}
}
}
long lastBackClick = SystemClock.elapsedRealtime();
@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
// If it wasn't the Back key or there's no web page history, bubble up to the default
// system behavior (probably exit the activity)
if (SystemClock.elapsedRealtime() - lastBackClick > 2000){
lastBackClick = SystemClock.elapsedRealtime();
Toast.makeText(this, "Click again to close the app",
Toast.LENGTH_LONG).show();
return true;
}
lastBackClick = SystemClock.elapsedRealtime();
return super.onKeyDown(keyCode, event);
}
//----------------------------------------------------------------------------
// Listener interface for onNewIntent
//
public interface NewIntentListener {
void onNewIntent(Intent intent);
}
private List<NewIntentListener> newIntentListeners = null;
public void registerNewIntentListener(NewIntentListener listener) {
if ( this.newIntentListeners == null )
this.newIntentListeners = Collections.synchronizedList(new ArrayList<NewIntentListener>());
this.newIntentListeners.add(listener);
}
public void unregisterNewIntentListener(NewIntentListener listener) {
if ( this.newIntentListeners == null )
return;
this.newIntentListeners.remove(listener);
}
@Override
protected void onNewIntent(Intent intent) {
if ( this.newIntentListeners == null )
return;
this.onResume();
synchronized ( this.newIntentListeners ) {
Iterator<NewIntentListener> iterator = this.newIntentListeners.iterator();
while ( iterator.hasNext() ) {
(iterator.next()).onNewIntent(intent);
}
}
}
//----------------------------------------------------------------------------
// Listener interface for onActivityResult
//
public interface ActivityResultListener {
void onActivityResult(int requestCode, int resultCode, Intent data);
}
private List<ActivityResultListener> activityResultListeners = null;
public void registerActivityResultListener(ActivityResultListener listener) {
if ( this.activityResultListeners == null )
this.activityResultListeners = Collections.synchronizedList(new ArrayList<ActivityResultListener>());
this.activityResultListeners.add(listener);
}
public void unregisterActivityResultListener(ActivityResultListener listener) {
if ( this.activityResultListeners == null )
return;
this.activityResultListeners.remove(listener);
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent intent) {
if ( this.activityResultListeners == null )
return;
this.onResume();
synchronized ( this.activityResultListeners ) {
Iterator<ActivityResultListener> iterator = this.activityResultListeners.iterator();
while ( iterator.hasNext() )
(iterator.next()).onActivityResult(requestCode, resultCode, intent);
}
}
public static void start_service(String serviceTitle, String serviceDescription,
String pythonServiceArgument) {
Intent serviceIntent = new Intent(PythonActivity.mActivity, PythonService.class);
String argument = PythonActivity.mActivity.getFilesDir().getAbsolutePath();
String filesDirectory = argument;
String app_root_dir = PythonActivity.mActivity.getAppRoot();
serviceIntent.putExtra("androidPrivate", argument);
serviceIntent.putExtra("androidArgument", app_root_dir);
serviceIntent.putExtra("serviceEntrypoint", "service/main.pyo");
serviceIntent.putExtra("pythonName", "python");
serviceIntent.putExtra("pythonHome", app_root_dir);
serviceIntent.putExtra("pythonPath", app_root_dir + ":" + app_root_dir + "/lib");
serviceIntent.putExtra("serviceTitle", serviceTitle);
serviceIntent.putExtra("serviceDescription", serviceDescription);
serviceIntent.putExtra("pythonServiceArgument", pythonServiceArgument);
PythonActivity.mActivity.startService(serviceIntent);
}
public static void stop_service() {
Intent serviceIntent = new Intent(PythonActivity.mActivity, PythonService.class);
PythonActivity.mActivity.stopService(serviceIntent);
}
public static native void nativeSetEnv(String j_name, String j_value);
public static native int nativeInit(Object arguments);
}
class PythonMain implements Runnable {
@Override
public void run() {
PythonActivity.nativeInit(new String[0]);
}
}
@@ -0,0 +1,45 @@
package org.kivy.android.concurrency;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
/**
* Created by ryan on 3/28/14.
*/
public class PythonEvent {
private final Lock lock = new ReentrantLock();
private final Condition cond = lock.newCondition();
private boolean flag = false;
public void set() {
lock.lock();
try {
flag = true;
cond.signalAll();
} finally {
lock.unlock();
}
}
public void wait_() throws InterruptedException {
lock.lock();
try {
while (!flag) {
cond.await();
}
} finally {
lock.unlock();
}
}
public void clear() {
lock.lock();
try {
flag = false;
cond.signalAll();
} finally {
lock.unlock();
}
}
}
@@ -0,0 +1,19 @@
package org.kivy.android.concurrency;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
/**
* Created by ryan on 3/28/14.
*/
public class PythonLock {
private final Lock lock = new ReentrantLock();
public void acquire() {
lock.lock();
}
public void release() {
lock.unlock();
}
}
@@ -0,0 +1,257 @@
package org.renpy.android;
import java.util.List;
import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;
import android.content.IntentFilter;
import android.hardware.Sensor;
import android.hardware.SensorEvent;
import android.hardware.SensorEventListener;
import android.hardware.SensorManager;
import android.net.ConnectivityManager;
import android.net.NetworkInfo;
import android.net.wifi.ScanResult;
import android.net.wifi.WifiManager;
import android.os.Vibrator;
import android.view.View;
/**
* Methods that are expected to be called via JNI, to access the device's
* non-screen hardware. (For example, the vibration and accelerometer.)
*/
public class Hardware {
// The context.
static Context context;
static View view;
/**
* Vibrate for s seconds.
*/
public static void vibrate(double s) {
Vibrator v = (Vibrator) context
.getSystemService(Context.VIBRATOR_SERVICE);
if (v != null) {
v.vibrate((int) (1000 * s));
}
}
/**
* Get an Overview of all Hardware Sensors of an Android Device
*/
public static String getHardwareSensors() {
SensorManager sm = (SensorManager) context
.getSystemService(Context.SENSOR_SERVICE);
List<Sensor> allSensors = sm.getSensorList(Sensor.TYPE_ALL);
if (allSensors != null) {
String resultString = "";
for (Sensor s : allSensors) {
resultString += String.format("Name=" + s.getName());
resultString += String.format(",Vendor=" + s.getVendor());
resultString += String.format(",Version=" + s.getVersion());
resultString += String.format(",MaximumRange="
+ s.getMaximumRange());
// XXX MinDelay is not in the 2.2
// resultString += String.format(",MinDelay=" +
// s.getMinDelay());
resultString += String.format(",Power=" + s.getPower());
resultString += String.format(",Type=" + s.getType() + "\n");
}
return resultString;
}
return "";
}
/**
* Get Access to 3 Axis Hardware Sensors Accelerometer, Orientation and
* Magnetic Field Sensors
*/
public static class generic3AxisSensor implements SensorEventListener {
private final SensorManager sSensorManager;
private final Sensor sSensor;
private final int sSensorType;
SensorEvent sSensorEvent;
public generic3AxisSensor(int sensorType) {
sSensorType = sensorType;
sSensorManager = (SensorManager) context
.getSystemService(Context.SENSOR_SERVICE);
sSensor = sSensorManager.getDefaultSensor(sSensorType);
}
public void onAccuracyChanged(Sensor sensor, int accuracy) {
}
public void onSensorChanged(SensorEvent event) {
sSensorEvent = event;
}
/**
* Enable or disable the Sensor by registering/unregistering
*/
public void changeStatus(boolean enable) {
if (enable) {
sSensorManager.registerListener(this, sSensor,
SensorManager.SENSOR_DELAY_NORMAL);
} else {
sSensorManager.unregisterListener(this, sSensor);
}
}
/**
* Read the Sensor
*/
public float[] readSensor() {
if (sSensorEvent != null) {
return sSensorEvent.values;
} else {
float rv[] = { 0f, 0f, 0f };
return rv;
}
}
}
public static generic3AxisSensor accelerometerSensor = null;
public static generic3AxisSensor orientationSensor = null;
public static generic3AxisSensor magneticFieldSensor = null;
/**
* functions for backward compatibility reasons
*/
public static void accelerometerEnable(boolean enable) {
if (accelerometerSensor == null)
accelerometerSensor = new generic3AxisSensor(
Sensor.TYPE_ACCELEROMETER);
accelerometerSensor.changeStatus(enable);
}
public static float[] accelerometerReading() {
float rv[] = { 0f, 0f, 0f };
if (accelerometerSensor == null)
return rv;
return (float[]) accelerometerSensor.readSensor();
}
public static void orientationSensorEnable(boolean enable) {
if (orientationSensor == null)
orientationSensor = new generic3AxisSensor(Sensor.TYPE_ORIENTATION);
orientationSensor.changeStatus(enable);
}
public static float[] orientationSensorReading() {
float rv[] = { 0f, 0f, 0f };
if (orientationSensor == null)
return rv;
return (float[]) orientationSensor.readSensor();
}
public static void magneticFieldSensorEnable(boolean enable) {
if (magneticFieldSensor == null)
magneticFieldSensor = new generic3AxisSensor(
Sensor.TYPE_MAGNETIC_FIELD);
magneticFieldSensor.changeStatus(enable);
}
public static float[] magneticFieldSensorReading() {
float rv[] = { 0f, 0f, 0f };
if (magneticFieldSensor == null)
return rv;
return (float[]) magneticFieldSensor.readSensor();
}
/**
* Scan WiFi networks
*/
static List<ScanResult> latestResult;
public static void enableWifiScanner() {
IntentFilter i = new IntentFilter();
i.addAction(WifiManager.SCAN_RESULTS_AVAILABLE_ACTION);
context.registerReceiver(new BroadcastReceiver() {
@Override
public void onReceive(Context c, Intent i) {
// Code to execute when SCAN_RESULTS_AVAILABLE_ACTION event
// occurs
WifiManager w = (WifiManager) c
.getSystemService(Context.WIFI_SERVICE);
latestResult = w.getScanResults(); // Returns a <list> of
// scanResults
}
}, i);
}
public static String scanWifi() {
// Now you can call this and it should execute the broadcastReceiver's
// onReceive()
WifiManager wm = (WifiManager) context
.getSystemService(Context.WIFI_SERVICE);
boolean a = wm.startScan();
if (latestResult != null) {
String latestResultString = "";
for (ScanResult result : latestResult) {
latestResultString += String.format("%s\t%s\t%d\n",
result.SSID, result.BSSID, result.level);
}
return latestResultString;
}
return "";
}
/**
* network state
*/
public static boolean network_state = false;
/**
* Check network state directly
*
* (only one connection can be active at a given moment, detects all network
* type)
*
*/
public static boolean checkNetwork() {
boolean state = false;
final ConnectivityManager conMgr = (ConnectivityManager) context
.getSystemService(Context.CONNECTIVITY_SERVICE);
final NetworkInfo activeNetwork = conMgr.getActiveNetworkInfo();
if (activeNetwork != null && activeNetwork.isConnected()) {
state = true;
} else {
state = false;
}
return state;
}
/**
* To recieve network state changes
*/
public static void registerNetworkCheck() {
IntentFilter i = new IntentFilter();
i.addAction(ConnectivityManager.CONNECTIVITY_ACTION);
context.registerReceiver(new BroadcastReceiver() {
@Override
public void onReceive(Context c, Intent i) {
network_state = checkNetwork();
}
}, i);
}
}
@@ -0,0 +1,93 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="{{ args.package }}"
android:versionCode="{{ args.numeric_version }}"
android:versionName="{{ args.version }}"
android:installLocation="auto">
<supports-screens
android:smallScreens="true"
android:normalScreens="true"
android:largeScreens="true"
android:anyDensity="true"
{% if args.min_sdk_version >= 9 %}
android:xlargeScreens="true"
{% endif %}
/>
<!-- Android 2.3.3 -->
<uses-sdk android:minSdkVersion="{{ args.min_sdk_version }}" android:targetSdkVersion="{{ android_api }}" />
<!-- Set permissions -->
{% for perm in args.permissions %}
{% if '.' in perm %}
<uses-permission android:name="{{ perm }}" />
{% else %}
<uses-permission android:name="android.permission.{{ perm }}" />
{% endif %}
{% endfor %}
{% if args.wakelock %}
<uses-permission android:name="android.permission.WAKE_LOCK" />
{% endif %}
{% if args.billing_pubkey %}
<uses-permission android:name="com.android.vending.BILLING" />
{% endif %}
<!-- Create a Java class extending SDLActivity and place it in a
directory under src matching the package, e.g.
src/com/gamemaker/game/MyGame.java
then replace "SDLActivity" with the name of your class (e.g. "MyGame")
in the XML below.
An example Java class can be found in README-android.txt
-->
<application android:label="@string/app_name"
android:icon="@drawable/icon"
android:allowBackup="true"
android:theme="@android:style/Theme.NoTitleBar{% if not args.window %}.Fullscreen{% endif %}"
android:hardwareAccelerated="true" >
{% for m in args.meta_data %}
<meta-data android:name="{{ m.split('=', 1)[0] }}" android:value="{{ m.split('=', 1)[-1] }}"/>{% endfor %}
<meta-data android:name="wakelock" android:value="{% if args.wakelock %}1{% else %}0{% endif %}"/>
<activity android:name="org.kivy.android.PythonActivity"
android:label="@string/app_name"
android:configChanges="keyboardHidden|orientation{% if args.min_sdk_version >= 13 %}|screenSize{% endif %}"
>
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
{%- if args.intent_filters -%}
{{- args.intent_filters -}}
{%- endif -%}
</activity>
{% if service %}
<service android:name="org.kivy.android.PythonService"
android:process=":pythonservice" />
{% endif %}
{% for name in service_names %}
<service android:name="{{ args.package }}.Service{{ name|capitalize }}"
android:process=":service_{{ name }}" />
{% endfor %}
{% if args.billing_pubkey %}
<service android:name="org.kivy.android.billing.BillingReceiver"
android:process=":pythonbilling" />
<receiver android:name="org.kivy.android.billing.BillingReceiver"
android:process=":pythonbillingreceiver">
<intent-filter>
<action android:name="com.android.vending.billing.IN_APP_NOTIFY" />
<action android:name="com.android.vending.billing.RESPONSE_CODE" />
<action android:name="com.android.vending.billing.PURCHASE_STATE_CHANGED" />
</intent-filter>
</receiver>
{% endif %}
</application>
</manifest>
@@ -0,0 +1,74 @@
package {{ args.package }};
import android.os.Binder;
import android.os.IBinder;
import android.content.Intent;
import android.content.Context;
import org.kivy.android.PythonService;
public class Service{{ name|capitalize }} extends PythonService {
/**
* Binder given to clients
*/
private final IBinder mBinder = new Service{{ name|capitalize }}Binder();
{% if sticky %}
/**
* {@inheritDoc}
*/
@Override
public int getStartType() {
return START_STICKY;
}
{% endif %}
{% if foreground %}
/**
* {@inheritDoc}
*/
@Override
public boolean getStartForeground() {
return true;
}
{% endif %}
public static void start(Context ctx, String pythonServiceArgument) {
String argument = ctx.getFilesDir().getAbsolutePath() + "/app";
Intent intent = new Intent(ctx, Service{{ name|capitalize }}.class);
intent.putExtra("androidPrivate", argument);
intent.putExtra("androidArgument", argument);
intent.putExtra("serviceEntrypoint", "{{ entrypoint }}");
intent.putExtra("serviceTitle", "{{ name|capitalize }}");
intent.putExtra("serviceDescription", "");
intent.putExtra("pythonName", "{{ name }}");
intent.putExtra("pythonHome", argument);
intent.putExtra("androidUnpack", argument);
intent.putExtra("pythonPath", argument + ":" + argument + "/lib");
intent.putExtra("pythonServiceArgument", pythonServiceArgument);
ctx.startService(intent);
}
public static void stop(Context ctx) {
Intent intent = new Intent(ctx, Service{{ name|capitalize }}.class);
ctx.stopService(intent);
}
/**
* {@inheritDoc}
*/
@Override
public IBinder onBind(Intent intent) {
return mBinder;
}
/**
* Class used for the client Binder. Because we know this service always
* runs in the same process as its clients, we don't need to deal with IPC.
*/
public class Service{{ name|capitalize }}Binder extends Binder {
Service{{ name|capitalize }} getService() {
// Return this instance of Service{{ name|capitalize }} so clients can call public methods
return Service{{ name|capitalize }}.this;
}
}
}
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="app_name">{{ args.name }}</string>
<string name="private_version">{{ private_version }}</string>
</resources>
+68
View File
@@ -0,0 +1,68 @@
from lbrynet.extras.daemon.Daemon import Daemon
from jnius import PythonJavaClass, autoclass, java_method
JAVA_NAMESPACE = 'org.kivy.android'
JNI_NAMESPACE = 'org/kivy/android'
_activity = autoclass(JAVA_NAMESPACE + '.PythonActivity').mActivity
_callbacks = {
'on_new_intent': [],
'on_activity_result': [],
}
class NewIntentListener(PythonJavaClass):
__javainterfaces__ = [JNI_NAMESPACE + '/PythonActivity$NewIntentListener']
__javacontext__ = 'app'
def __init__(self, callback, **kwargs):
super(NewIntentListener, self).__init__(**kwargs)
self.callback = callback
@java_method('(Landroid/content/Intent;)V')
def onNewIntent(self, intent):
self.callback(intent)
class ActivityResultListener(PythonJavaClass):
__javainterfaces__ = [JNI_NAMESPACE + '/PythonActivity$ActivityResultListener']
__javacontext__ = 'app'
def __init__(self, callback):
super(ActivityResultListener, self).__init__()
self.callback = callback
@java_method('(IILandroid/content/Intent;)V')
def onActivityResult(self, requestCode, resultCode, intent):
self.callback(requestCode, resultCode, intent)
def bind(**kwargs):
for event, callback in kwargs.items():
if event not in _callbacks:
raise Exception('Unknown {!r} event'.format(event))
elif event == 'on_new_intent':
listener = NewIntentListener(callback)
_activity.registerNewIntentListener(listener)
_callbacks[event].append(listener)
elif event == 'on_activity_result':
listener = ActivityResultListener(callback)
_activity.registerActivityResultListener(listener)
_callbacks[event].append(listener)
def unbind(**kwargs):
for event, callback in kwargs.items():
if event not in _callbacks:
raise Exception('Unknown {!r} event'.format(event))
else:
for listener in _callbacks[event][:]:
if listener.callback == callback:
_callbacks[event].remove(listener)
if event == 'on_new_intent':
_activity.unregisterNewIntentListener(listener)
elif event == 'on_activity_result':
_activity.unregisterActivityResultListener(listener)
print('YEAH! Done all the things!!! WOOOHOOO!!!!')
+49
View File
@@ -0,0 +1,49 @@
from os.path import join, dirname, abspath
from pythonforandroid.toolchain import Bootstrap
from setuptools import setup, find_packages
from bootstrap import LBRYServiceBootstrap
Bootstrap.bootstraps = {
'lbry-service': LBRYServiceBootstrap()
}
setup(
name='lbryservice',
version='0.1',
author="LBRY Inc.",
author_email="hello@lbry.io",
url="https://lbry.io",
description="Android Service for LBRY Network.",
license='MIT',
python_requires='>=3.7',
packages=find_packages(),
package_data={'service': ['*.py']},
options={
'apk': {
'dist_name': 'lbry-service',
'bootstrap': 'lbry-service',
'package': 'io.lbry.service',
'permissions': ['INTERNET'],
'requirements': ','.join([
# needed by aiohttp
'multidict', 'yarl', 'async_timeout', 'chardet',
# minimum needed by torba:
'aiohttp', 'coincurve', 'pbkdf2', 'cryptography', 'attrs',
abspath(join(dirname(__file__), '..', '..', 'torba')),
# minimum needed by lbrynet
'aioupnp', 'appdirs', 'distro', 'base58', 'jsonrpc', 'protobuf',
'msgpack', 'jsonschema', 'ecdsa', 'pyyaml', 'docopt',
abspath(join(dirname(__file__), '..')),
'genericndkbuild', 'pyjnius', 'sqlite3', 'python3'
]),
'android-api': '26',
'ndk-api': '21',
'ndk-version': 'r17c',
'arch': 'armeabi-v7a',
'sdk-dir': '/home/lex/projects/android',
'ndk-dir': '/home/lex/projects/android/android-ndk-r17c/'
}
}
)
-33
View File
@@ -1,33 +0,0 @@
$env:Path += ";C:\MinGW\bin\"
$env:Path += ";C:\Program Files (x86)\Windows Kits\10\bin\x86\"
gcc --version
mingw32-make --version
# build/install miniupnpc manually
tar zxf miniupnpc-1.9.tar.gz
cd miniupnpc-1.9
mingw32-make -f Makefile.mingw
python setupmingw32.py build --compiler=mingw32
python setupmingw32.py install
cd ..\
Remove-Item -Recurse -Force miniupnpc-1.9
# copy requirements from lbry, but remove miniupnpc (installed manually)
Get-Content ..\requirements.txt | Select-String -Pattern 'miniupnpc' -NotMatch | Out-File requirements_base.txt
python set_build.py
pip install -r requirements.txt
pip install ..\.
pyinstaller -y daemon.onefile.spec
pyinstaller -y cli.onefile.spec
pyinstaller -y console.onefile.spec
nuget install secure-file -ExcludeVersion
secure-file\tools\secure-file -decrypt .\lbry2.pfx.enc -secret "$env:pfx_key"
signtool.exe sign /f .\lbry2.pfx /p "$env:key_pass" /tr http://tsa.starfieldtech.com /td SHA256 /fd SHA256 dist\*.exe
python zip_daemon.py
python upload_assets.py
-59
View File
@@ -1,59 +0,0 @@
#!/bin/bash
set -euo pipefail
set -x
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )"
cd "$ROOT"
BUILD_DIR="$ROOT/build"
FULL_BUILD="${FULL_BUILD:-false}"
if [ -n "${TEAMCITY_VERSION:-}" -o -n "${APPVEYOR:-}" ]; then
FULL_BUILD="true"
fi
[ -d "$BUILD_DIR/bulid" ] && rm -rf "$BUILD_DIR/build"
[ -d "$BUILD_DIR/dist" ] && rm -rf "$BUILD_DIR/dist"
if [ "$FULL_BUILD" == "true" ]; then
# install dependencies
$BUILD_DIR/prebuild.sh
VENV="$BUILD_DIR/venv"
if [ -d "$VENV" ]; then
rm -rf "$VENV"
fi
virtualenv "$VENV"
set +u
source "$VENV/bin/activate"
set -u
# must set build before installing lbrynet. otherwise it has no effect
python "$BUILD_DIR/set_build.py"
fi
cp "$ROOT/requirements.txt" "$BUILD_DIR/requirements_base.txt"
(
cd "$BUILD_DIR"
pip install -r requirements.txt
)
(
cd "$BUILD_DIR"
pyinstaller -y daemon.onefile.spec
pyinstaller -y cli.onefile.spec
pyinstaller -y console.onefile.spec
)
python "$BUILD_DIR/zip_daemon.py"
if [ "$FULL_BUILD" == "true" ]; then
# electron-build has a publish feature, but I had a hard time getting
# it to reliably work and it also seemed difficult to configure. Not proud of
# this, but it seemed better to write my own.
python "$BUILD_DIR/upload_assets.py"
deactivate
fi
echo 'Build complete.'
-42
View File
@@ -1,42 +0,0 @@
# -*- mode: python -*-
import platform
import os
dir = 'build';
cwd = os.getcwd()
if os.path.basename(cwd) != dir:
raise Exception('pyinstaller build needs to be run from the ' + dir + ' directory')
repo_base = os.path.abspath(os.path.join(cwd, '..'))
execfile(os.path.join(cwd, "entrypoint.py")) # ghetto import
system = platform.system()
if system == 'Darwin':
icns = os.path.join(repo_base, 'build', 'icon.icns')
elif system == 'Linux':
icns = os.path.join(repo_base, 'build', 'icons', '256x256.png')
elif system == 'Windows':
icns = os.path.join(repo_base, 'build', 'icons', 'lbry256.ico')
else:
print 'Warning: System {} has no icons'.format(system)
icns = None
a = Entrypoint('lbrynet', 'console_scripts', 'lbrynet-cli', pathex=[cwd])
pyz = PYZ(a.pure, a.zipped_data)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='lbrynet-cli',
debug=False,
strip=False,
upx=True,
console=True,
icon=icns
)
-50
View File
@@ -1,50 +0,0 @@
# -*- mode: python -*-
import platform
import os
import lbryum
dir = 'build';
cwd = os.getcwd()
if os.path.basename(cwd) != dir:
raise Exception('pyinstaller build needs to be run from the ' + dir + ' directory')
repo_base = os.path.abspath(os.path.join(cwd, '..'))
execfile(os.path.join(cwd, "entrypoint.py")) # ghetto import
system = platform.system()
if system == 'Darwin':
icns = os.path.join(repo_base, 'build', 'icon.icns')
elif system == 'Linux':
icns = os.path.join(repo_base, 'build', 'icons', '256x256.png')
elif system == 'Windows':
icns = os.path.join(repo_base, 'build', 'icons', 'lbry256.ico')
else:
print 'Warning: System {} has no icons'.format(system)
icns = None
datas = [
(os.path.join(os.path.dirname(lbryum.__file__), 'wordlist', language + '.txt'), 'lbryum/wordlist')
for language in ('chinese_simplified', 'japanese', 'spanish','english', 'portuguese')
]
a = Entrypoint('lbrynet', 'console_scripts', 'lbrynet-console', pathex=[cwd], datas=datas)
pyz = PYZ(a.pure, a.zipped_data)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='lbrynet-console',
debug=False,
strip=False,
upx=True,
console=True,
icon=icns
)
-50
View File
@@ -1,50 +0,0 @@
# -*- mode: python -*-
import platform
import os
import lbryum
dir = 'build';
cwd = os.getcwd()
if os.path.basename(cwd) != dir:
raise Exception('pyinstaller build needs to be run from the ' + dir + ' directory')
repo_base = os.path.abspath(os.path.join(cwd, '..'))
execfile(os.path.join(cwd, "entrypoint.py")) # ghetto import
system = platform.system()
if system == 'Darwin':
icns = os.path.join(repo_base, 'build', 'icon.icns')
elif system == 'Linux':
icns = os.path.join(repo_base, 'build', 'icons', '256x256.png')
elif system == 'Windows':
icns = os.path.join(repo_base, 'build', 'icons', 'lbry256.ico')
else:
print 'Warning: System {} has no icons'.format(system)
icns = None
datas = [
(os.path.join(os.path.dirname(lbryum.__file__), 'wordlist', language + '.txt'), 'lbryum/wordlist')
for language in ('chinese_simplified', 'japanese', 'spanish','english', 'portuguese')
]
a = Entrypoint('lbrynet', 'console_scripts', 'lbrynet-daemon', pathex=[cwd], datas=datas)
pyz = PYZ(a.pure, a.zipped_data)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='lbrynet-daemon',
debug=False,
strip=False,
upx=True,
console=True,
icon=icns
)
-47
View File
@@ -1,47 +0,0 @@
# https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Setuptools-Entry-Point
def Entrypoint(dist, group, name,
scripts=None, pathex=None, binaries=None, datas=None,
hiddenimports=None, hookspath=None, excludes=None, runtime_hooks=None,
cipher=None, win_no_prefer_redirects=False, win_private_assemblies=False):
import pkg_resources
# get toplevel packages of distribution from metadata
def get_toplevel(dist):
distribution = pkg_resources.get_distribution(dist)
if distribution.has_metadata('top_level.txt'):
return list(distribution.get_metadata('top_level.txt').split())
else:
return []
hiddenimports = hiddenimports or []
packages = []
for distribution in hiddenimports:
packages += get_toplevel(distribution)
scripts = scripts or []
pathex = pathex or []
# get the entry point
ep = pkg_resources.get_entry_info(dist, group, name)
# insert path of the egg at the verify front of the search path
pathex = [ep.dist.location] + pathex
# script name must not be a valid module name to avoid name clashes on import
script_path = os.path.join(workpath, name + '-script.py')
print "creating script for entry point", dist, group, name
with open(script_path, 'w') as fh:
fh.write("import {0}\n".format(ep.module_name))
fh.write("{0}.{1}()\n".format(ep.module_name, '.'.join(ep.attrs)))
for package in packages:
fh.write("import {0}\n".format(package))
return Analysis([script_path] + scripts,
pathex=pathex,
binaries=binaries,
datas=datas,
hiddenimports=hiddenimports,
hookspath=hookspath,
excludes=excludes,
runtime_hooks=runtime_hooks,
cipher=cipher,
win_no_prefer_redirects=win_no_prefer_redirects,
win_private_assemblies=win_private_assemblies
)
Binary file not shown.
Binary file not shown.
-81
View File
@@ -1,81 +0,0 @@
#!/bin/bash
set -euo pipefail
set -x
LINUX=false
OSX=false
if [ "$(uname)" == "Darwin" ]; then
OSX=true
elif [ "$(expr substr $(uname -s) 1 5)" == "Linux" ]; then
LINUX=true
else
echo "Platform detection failed"
exit 1
fi
SUDO=''
if $LINUX && (( $EUID != 0 )); then
SUDO='sudo'
fi
cmd_exists() {
command -v "$1" >/dev/null 2>&1
return $?
}
set +eu
GITUSERNAME=$(git config --global --get user.name)
if [ -z "$GITUSERNAME" ]; then
git config --global user.name "$(whoami)"
fi
GITEMAIL=$(git config --global --get user.email)
if [ -z "$GITEMAIL" ]; then
git config --global user.email "$(whoami)@lbry.io"
fi
set -eu
if $LINUX; then
INSTALL="$SUDO apt-get install --no-install-recommends -y"
$INSTALL build-essential libssl-dev libffi-dev python2.7-dev wget
elif $OSX && ! cmd_exists brew ; then
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
fi
if ! cmd_exists python; then
if $LINUX; then
$INSTALL python2.7
elif $OSX; then
brew install python
fi
fi
PYTHON_VERSION=$(python -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')
if [ "$PYTHON_VERSION" != "2.7" ]; then
echo "Python 2.7 required"
exit 1
fi
if ! cmd_exists pip; then
if $LINUX; then
$INSTALL python-pip
$SUDO pip install --upgrade pip
else
echo "Pip required"
exit 1
fi
fi
if $LINUX && [ "$(pip list --format=columns | grep setuptools | wc -l)" -ge 1 ]; then
#$INSTALL python-setuptools
$SUDO pip install setuptools
fi
if ! cmd_exists virtualenv; then
$SUDO pip install virtualenv
fi
-11
View File
@@ -1,11 +0,0 @@
# install daemon requirements (created by build script. see build.sh, build.ps1)
-r requirements_base.txt
# install daemon itself. make sure you run `pip install` from this dir. this is how you do relative file paths with pip
file:../.
# install other build requirements
PyInstaller==3.2.1
requests[security]==2.13.0
uritemplate==3.0.0
boto3==1.4.4
-143
View File
@@ -1,143 +0,0 @@
import glob
import json
import os
import subprocess
import sys
import github
import uritemplate
import boto3
def main():
upload_to_github_if_tagged('lbryio/lbry')
upload_to_s3('daemon')
def get_asset_filename():
this_dir = os.path.dirname(os.path.realpath(__file__))
return glob.glob(this_dir + '/dist/*.zip')[0]
def upload_to_s3(folder):
tag = subprocess.check_output(['git', 'describe', '--always', '--abbrev=8', 'HEAD']).strip()
commit_date = subprocess.check_output([
'git', 'show', '-s', '--format=%cd', '--date=format:%Y%m%d-%H%I%S', 'HEAD']).strip()
asset_path = get_asset_filename()
bucket = 'releases.lbry.io'
key = folder + '/' + commit_date + '-' + tag + '/' + os.path.basename(asset_path)
print "Uploading " + asset_path + " to s3://" + bucket + '/' + key + ''
if 'AWS_ACCESS_KEY_ID' not in os.environ or 'AWS_SECRET_ACCESS_KEY' not in os.environ:
print 'Must set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to publish assets to s3'
return 1
s3 = boto3.resource(
's3',
aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'],
aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
config=boto3.session.Config(signature_version='s3v4')
)
s3.meta.client.upload_file(asset_path, bucket, key)
def upload_to_github_if_tagged(repo_name):
try:
current_tag = subprocess.check_output(
['git', 'describe', '--exact-match', 'HEAD']).strip()
except subprocess.CalledProcessError:
print 'Not uploading to GitHub as we are not currently on a tag'
return 1
print "Current tag: " + current_tag
if 'GH_TOKEN' not in os.environ:
print 'Must set GH_TOKEN in order to publish assets to a release'
return 1
gh_token = os.environ['GH_TOKEN']
auth = github.Github(gh_token)
repo = auth.get_repo(repo_name)
if not check_repo_has_tag(repo, current_tag):
print 'Tag {} is not in repo {}'.format(current_tag, repo)
# TODO: maybe this should be an error
return 1
asset_path = get_asset_filename()
print "Uploading " + asset_path + " to Github tag " + current_tag
release = get_github_release(repo, current_tag)
upload_asset_to_github(release, asset_path, gh_token)
def check_repo_has_tag(repo, target_tag):
tags = repo.get_tags().get_page(0)
for tag in tags:
if tag.name == target_tag:
return True
return False
def get_github_release(repo, current_tag):
for release in repo.get_releases():
if release.tag_name == current_tag:
return release
raise Exception('No release for {} was found'.format(current_tag))
def upload_asset_to_github(release, asset_to_upload, token):
basename = os.path.basename(asset_to_upload)
for asset in release.raw_data['assets']:
if asset['name'] == basename:
print 'File {} has already been uploaded to {}'.format(basename, release.tag_name)
return
upload_uri = uritemplate.expand(release.upload_url, {'name': basename})
count = 0
while count < 10:
try:
output = _curl_uploader(upload_uri, asset_to_upload, token)
if 'errors' in output:
raise Exception(output)
else:
print 'Successfully uploaded to {}'.format(output['browser_download_url'])
except Exception:
print 'Failed uploading on attempt {}'.format(count + 1)
count += 1
def _curl_uploader(upload_uri, asset_to_upload, token):
# using requests.post fails miserably with SSL EPIPE errors. I spent
# half a day trying to debug before deciding to switch to curl.
#
# TODO: actually set the content type
print 'Using curl to upload {} to {}'.format(asset_to_upload, upload_uri)
cmd = [
'curl',
'-sS',
'-X', 'POST',
'-u', ':{}'.format(os.environ['GH_TOKEN']),
'--header', 'Content-Type: application/octet-stream',
'--data-binary', '@-',
upload_uri
]
# '-d', '{"some_key": "some_value"}',
print 'Calling curl:'
print cmd
print
with open(asset_to_upload, 'rb') as fp:
p = subprocess.Popen(cmd, stdin=fp, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
stdout, stderr = p.communicate()
print 'curl return code:', p.returncode
if stderr:
print 'stderr output from curl:'
print stderr
print 'stdout from curl:'
print stdout
return json.loads(stdout)
if __name__ == '__main__':
sys.exit(main())
-29
View File
@@ -1,29 +0,0 @@
import os
import platform
import subprocess
import sys
import zipfile
def main():
this_dir = os.path.dirname(os.path.realpath(__file__))
tag = subprocess.check_output(['git', 'describe']).strip()
zipfilename = 'lbrynet-daemon-{}-{}.zip'.format(tag, get_system_label())
full_filename = os.path.join(this_dir, 'dist', zipfilename)
executables = ['lbrynet-daemon', 'lbrynet-cli', 'lbrynet-console']
ext = '.exe' if platform.system() == 'Windows' else ''
with zipfile.ZipFile(full_filename, 'w') as myzip:
for executable in executables:
myzip.write(os.path.join(this_dir, 'dist', executable + ext), executable + ext)
def get_system_label():
system = platform.system()
if system == 'Darwin':
return 'macos'
else:
return system.lower()
if __name__ == '__main__':
sys.exit(main())
+5 -5
View File
@@ -32,7 +32,7 @@
<meta name="lang:search.tokenizer" content="[\s\-]+">
<link rel="shortcut icon" href="/assets/images/favicon.png">
<meta name="generator" content="mkdocs-0.17.2, mkdocs-material-2.7.0">
<meta name="generator" content="mkdocs-0.17.3, mkdocs-material-2.7.0">
@@ -100,7 +100,7 @@
<div class="md-flex__cell md-flex__cell--shrink">
<a href="/" title="LBRY" class="md-header-nav__button md-logo">
<img src="https://s3.amazonaws.com/files.lbry.io/logo-square-white-bookonly.png" width="24" height="24">
<img src="https://s3.amazonaws.com/files.lbry.io/logo-square-white-bookonly.png" alt="LBRY logo" width="24" height="24">
</a>
</div>
@@ -196,7 +196,7 @@
<label class="md-nav__title md-nav__title--site" for="drawer">
<span class="md-nav__button md-logo">
<img src="https://s3.amazonaws.com/files.lbry.io/logo-square-white-bookonly.png" width="48" height="48">
<img src="https://s3.amazonaws.com/files.lbry.io/logo-square-white-bookonly.png" alt="LBRY logo" width="48" height="48">
</span>
LBRY
@@ -295,7 +295,7 @@
<script src="/assets/javascripts/application.8eb9be28.js"></script>
<script>app.initialize({version:"0.17.2",url:{base:""}})</script>
<script>app.initialize({version:"0.17.3",url:{base:""}})</script>
@@ -304,4 +304,4 @@
</body>
</html>
</html>
+1605
View File
File diff suppressed because it is too large. Load diff
+192 -97
View File
@@ -32,7 +32,7 @@
<meta name="lang:search.tokenizer" content="[\s\-]+">
<link rel="shortcut icon" href="../assets/images/favicon.png">
<meta name="generator" content="mkdocs-0.17.2, mkdocs-material-2.7.0">
<meta name="generator" content="mkdocs-0.17.3, mkdocs-material-2.7.0">
@@ -307,6 +307,13 @@
blob_list
</a>
</li>
<li class="md-nav__item">
<a href="#blob_reflect" title="blob_reflect" class="md-nav__link">
blob_reflect
</a>
</li>
<li class="md-nav__item">
@@ -475,6 +482,13 @@
peer_list
</a>
</li>
<li class="md-nav__item">
<a href="#peer_ping" title="peer_ping" class="md-nav__link">
peer_ping
</a>
</li>
<li class="md-nav__item">
@@ -482,6 +496,13 @@
publish
</a>
</li>
<li class="md-nav__item">
<a href="#report_bug" title="report_bug" class="md-nav__link">
report_bug
</a>
</li>
<li class="md-nav__item">
@@ -709,6 +730,13 @@
blob_list
</a>
</li>
<li class="md-nav__item">
<a href="#blob_reflect" title="blob_reflect" class="md-nav__link">
blob_reflect
</a>
</li>
<li class="md-nav__item">
@@ -877,6 +905,13 @@
peer_list
</a>
</li>
<li class="md-nav__item">
<a href="#peer_ping" title="peer_ping" class="md-nav__link">
peer_ping
</a>
</li>
<li class="md-nav__item">
@@ -884,6 +919,13 @@
publish
</a>
</li>
<li class="md-nav__item">
<a href="#report_bug" title="report_bug" class="md-nav__link">
report_bug
</a>
</li>
<li class="md-nav__item">
@@ -1071,18 +1113,16 @@
<pre><code class="text">Announce blobs to the DHT
Usage:
blob_announce [--announce_all] [&lt;blob_hash&gt; | --blob_hash=&lt;blob_hash&gt;]
[&lt;stream_hash&gt; | --stream_hash=&lt;stream_hash&gt;]
[&lt;sd_hash&gt; | --sd_hash=&lt;sd_hash&gt;]
blob_announce [&lt;blob_hash&gt; | --blob_hash=&lt;blob_hash&gt;]
[&lt;stream_hash&gt; | --stream_hash=&lt;stream_hash&gt;] | [&lt;sd_hash&gt; | --sd_hash=&lt;sd_hash&gt;]
Options:
--announce_all=&lt;announce_all&gt; : (bool) announce all the blobs possessed by user
--blob_hash=&lt;blob_hash&gt; : (str) announce a blob, specified by blob_hash
--stream_hash=&lt;stream_hash&gt; : (str) announce all blobs associated with
stream_hash
--sd_hash=&lt;sd_hash&gt; : (str) announce all blobs associated with
sd_hash and the sd_hash itself
--blob_hash=&lt;blob_hash&gt; : (str) announce a blob, specified by blob_hash
--stream_hash=&lt;stream_hash&gt; : (str) announce all blobs associated with
stream_hash
--sd_hash=&lt;sd_hash&gt; : (str) announce all blobs associated with
sd_hash and the sd_hash itself
Returns:
(bool) true if successful
@@ -1172,6 +1212,20 @@ Returns:
(list) List of blob hashes
</code></pre>
<h2 id="blob_reflect">blob_reflect</h2>
<pre><code class="text">Reflects specified blobs
Usage:
blob_reflect (&lt;blob_hashes&gt;...) [--reflector_server=&lt;reflector_server&gt;]
Options:
--reflector_server=&lt;reflector_server&gt; : (str) reflector address
Returns:
(list) reflected blob hashes
</code></pre>
<h2 id="blob_reflect_all">blob_reflect_all</h2>
<pre><code class="text">Reflects all saved blobs
@@ -1219,8 +1273,7 @@ Returns:
<pre><code class="text">Import serialized channel signing information (to allow signing new claims to the channel)
Usage:
channel_import (&lt;serialized_certificate_info&gt; |
--serialized_certificate_info=&lt;serialized_certificate_info&gt;)
channel_import (&lt;serialized_certificate_info&gt; | --serialized_certificate_info=&lt;serialized_certificate_info&gt;)
Options:
@@ -1401,7 +1454,7 @@ Returns:
'is_spent': (bool) true if claim is abandoned, false otherwise
'name': (str) name of the claim
'permanent_url': (str) permanent url of the claim,
'txid': (str) txid of the cliam
'txid': (str) txid of the claim
'nout': (int) nout of the claim
'value': (str) value of the claim
},
@@ -1526,12 +1579,12 @@ Usage:
Options:
--a_arg : a arg
--b_arg : b arg
--pos_arg=&lt;pos_arg&gt; : pos arg
--pos_args=&lt;pos_args&gt; : pos args
--pos_arg2=&lt;pos_arg2&gt; : pos arg 2
--pos_arg3=&lt;pos_arg3&gt; : pos arg 3
--a_arg : (bool) a arg
--b_arg : (bool) b arg
--pos_arg=&lt;pos_arg&gt; : (int) pos arg
--pos_args=&lt;pos_args&gt; : (int) pos args
--pos_arg2=&lt;pos_arg2&gt; : (int) pos arg 2
--pos_arg3=&lt;pos_arg3&gt; : (int) pos arg 3
Returns:
pos args
@@ -1603,7 +1656,7 @@ Usage:
file_list [--sd_hash=&lt;sd_hash&gt;] [--file_name=&lt;file_name&gt;] [--stream_hash=&lt;stream_hash&gt;]
[--rowid=&lt;rowid&gt;] [--claim_id=&lt;claim_id&gt;] [--outpoint=&lt;outpoint&gt;] [--txid=&lt;txid&gt;] [--nout=&lt;nout&gt;]
[--channel_claim_id=&lt;channel_claim_id&gt;] [--channel_name=&lt;channel_name&gt;]
[--claim_name=&lt;claim_name&gt;] [--full_status]
[--claim_name=&lt;claim_name&gt;] [--full_status] [--sort=&lt;sort_method&gt;...]
Options:
@@ -1621,6 +1674,9 @@ Options:
--claim_name=&lt;claim_name&gt; : (str) get file with matching claim name
--full_status : (bool) full status, populate the
'message' and 'size' fields
--sort=&lt;sort_method&gt; : (str) sort by any property, like 'file_name'
or 'metadata.author'; to specify direction
append ',asc' or ',desc'
Returns:
(list) List of files
@@ -1768,7 +1824,23 @@ Options:
--timeout=&lt;timeout&gt; : (int) peer search timeout in seconds
Returns:
(list) List of contacts
(list) List of contact dictionaries {'host': &lt;peer ip&gt;, 'port': &lt;peer port&gt;, 'node_id': &lt;peer node id&gt;}
</code></pre>
<h2 id="peer_ping">peer_ping</h2>
<pre><code class="text">Send a kademlia ping to the specified peer. If address and port are provided the peer is directly pinged,
if not provided the peer is located first.
Usage:
peer_ping (&lt;node_id&gt; | --node_id=&lt;node_id&gt;) [&lt;address&gt; | --address=&lt;address&gt;] [&lt;port&gt; | --port=&lt;port&gt;]
Options:
--address=&lt;address&gt; : (str) ip address of the peer
--port=&lt;port&gt; : (int) udp port of the peer
Returns:
(str) pong, or {'error': &lt;error message&gt;} if an error is encountered
</code></pre>
<h2 id="publish">publish</h2>
@@ -1798,40 +1870,40 @@ Usage:
Options:
--name=&lt;name&gt; : (str) name of the content
--bid=&lt;bid&gt; : (float) amount to back the claim
--metadata=&lt;metadata&gt; : (dict) ClaimDict to associate with the claim.
--file_path=&lt;file_path&gt; : (str) path to file to be associated with name. If provided,
a lbry stream of this file will be used in 'sources'.
If no path is given but a sources dict is provided,
it will be used. If neither are provided, an
error is raised.
--fee=&lt;fee&gt; : (dict) Dictionary representing key fee to download content:
{
'currency': currency_symbol,
'amount': float,
'address': str, optional
}
supported currencies: LBC, USD, BTC
If an address is not provided a new one will be
automatically generated. Default fee is zero.
--title=&lt;title&gt; : (str) title of the publication
--description=&lt;description&gt; : (str) description of the publication
--author=&lt;author&gt; : (str) author of the publication
--language=&lt;language&gt; : (str) language of the publication
--license=&lt;license&gt; : (str) publication license
--license_url=&lt;license_url&gt; : (str) publication license url
--thumbnail=&lt;thumbnail&gt; : (str) thumbnail url
--preview=&lt;preview&gt; : (str) preview url
--nsfw=&lt;nsfw&gt; : (bool) title of the publication
--sources=&lt;sources&gt; : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file
--channel_name=&lt;channel_name&gt; : (str) name of the publisher channel name in the wallet
--channel_id=&lt;channel_id&gt; : (str) claim id of the publisher channel, does not check
for channel claim being in the wallet. This allows
publishing to a channel where only the certificate
private key is in the wallet.
--claim_address=&lt;claim_address&gt; : (str) address where the claim is sent to, if not specified
new address wil automatically be created
--name=&lt;name&gt; : (str) name of the content
--bid=&lt;bid&gt; : (decimal) amount to back the claim
--metadata=&lt;metadata&gt; : (dict) ClaimDict to associate with the claim.
--file_path=&lt;file_path&gt; : (str) path to file to be associated with name. If provided,
a lbry stream of this file will be used in 'sources'.
If no path is given but a sources dict is provided,
it will be used. If neither are provided, an
error is raised.
--fee=&lt;fee&gt; : (dict) Dictionary representing key fee to download content:
{
'currency': currency_symbol,
'amount': decimal,
'address': str, optional
}
supported currencies: LBC, USD, BTC
If an address is not provided a new one will be
automatically generated. Default fee is zero.
--title=&lt;title&gt; : (str) title of the publication
--description=&lt;description&gt; : (str) description of the publication
--author=&lt;author&gt; : (str) author of the publication
--language=&lt;language&gt; : (str) language of the publication
--license=&lt;license&gt; : (str) publication license
--license_url=&lt;license_url&gt; : (str) publication license url
--thumbnail=&lt;thumbnail&gt; : (str) thumbnail url
--preview=&lt;preview&gt; : (str) preview url
--nsfw=&lt;nsfw&gt; : (bool) whether the content is nsfw
--sources=&lt;sources&gt; : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file
--channel_name=&lt;channel_name&gt; : (str) name of the publisher channel name in the wallet
--channel_id=&lt;channel_id&gt; : (str) claim id of the publisher channel, does not check
for channel claim being in the wallet. This allows
publishing to a channel where only the certificate
private key is in the wallet.
--claim_address=&lt;claim_address&gt; : (str) address where the claim is sent to, if not specified
new address will automatically be created
Returns:
(dict) Dictionary containing result of the claim
@@ -1839,11 +1911,25 @@ Returns:
'tx' : (str) hex encoded transaction
'txid' : (str) txid of resulting claim
'nout' : (int) nout of the resulting claim
'fee' : (float) fee paid for the claim transaction
'fee' : (decimal) fee paid for the claim transaction
'claim_id' : (str) claim ID of the resulting claim
}
</code></pre>
<h2 id="report_bug">report_bug</h2>
<pre><code class="text">Report a bug to slack
Usage:
report_bug (&lt;message&gt; | --message=&lt;message&gt;)
Options:
--message=&lt;message&gt; : (str) Description of the bug
Returns:
(bool) true if successful
</code></pre>
<h2 id="resolve">resolve</h2>
<pre><code class="text">Resolve given LBRY URIs
@@ -1945,6 +2031,7 @@ Returns:
&lt;bucket index&gt;: [
{
&quot;address&quot;: (str) peer address,
&quot;port&quot;: (int) peer udp port
&quot;node_id&quot;: (str) peer node id,
&quot;blobs&quot;: (list) blob hashes announced by peer
}
@@ -2025,54 +2112,62 @@ Returns:
<pre><code class="text">Get daemon status
Usage:
status [--session_status] [--dht_status]
status
Options:
--session_status : (bool) include session status in results
--dht_status : (bool) include dht network and peer status
None
Returns:
(dict) lbrynet-daemon status
{
'lbry_id': lbry peer id, base58,
'installation_id': installation id, base58,
'is_running': bool,
'installation_id': (str) installation id - base58,
'is_running': (bool),
'is_first_run': bool,
'startup_status': {
'code': status code,
'message': status message
'skipped_components': (list) [names of skipped components (str)],
'startup_status': { Does not include components which have been skipped
'database': (bool),
'wallet': (bool),
'session': (bool),
'dht': (bool),
'hash_announcer': (bool),
'stream_identifier': (bool),
'file_manager': (bool),
'blob_manager': (bool),
'blockchain_headers': (bool),
'peer_protocol_server': (bool),
'reflector': (bool),
'upnp': (bool),
'exchange_rate_manager': (bool),
},
'connection_status': {
'code': connection status code,
'message': connection status message
'code': (str) connection status code,
'message': (str) connection status message
},
'blockchain_status': {
'blocks': local blockchain height,
'blocks_behind': remote_height - local_height,
'best_blockhash': block hash of most recent block,
'blockchain_headers': {
'downloading_headers': (bool),
'download_progress': (float) 0-100.0
},
'wallet_is_encrypted': bool,
If given the session status option:
'session_status': {
'managed_blobs': count of blobs in the blob manager,
'managed_streams': count of streams in the file manager
'announce_queue_size': number of blobs currently queued to be announced
'should_announce_blobs': number of blobs that should be announced
}
If given the dht status option:
'dht_status': {
'kbps_received': current kbps receiving,
'kbps_sent': current kdps being sent,
'total_bytes_sent': total bytes sent,
'total_bytes_received': total bytes received,
'queries_received': number of queries received per second,
'queries_sent': number of queries sent per second,
'recent_contacts': count of recently contacted peers,
'unique_contacts': count of unique peers
},
'wallet': {
'blocks': (int) local blockchain height,
'blocks_behind': (int) remote_height - local_height,
'best_blockhash': (str) block hash of most recent block,
'is_encrypted': (bool),
'is_locked': (bool),
},
'dht': {
'node_id': (str) lbry dht node id - hex encoded,
'peers_in_routing_table': (int) the number of peers in the routing table,
},
'blob_manager': {
'finished_blobs': (int) number of finished blobs in the blob manager,
},
'hash_announcer': {
'announce_queue_size': (int) number of blobs currently queued to be announced
},
'file_manager': {
'managed_files': (int) count of files in the file manager,
}
}
</code></pre>
@@ -2089,7 +2184,7 @@ Options:
--uri=&lt;uri&gt; : (str) check availability for this uri
--search_timeout=&lt;search_timeout&gt; : (int) how long to search for peers for the blob
in the dht
--search_timeout=&lt;blob_timeout&gt; : (int) how long to try downloading from a peer
--blob_timeout=&lt;blob_timeout&gt; : (int) how long to try downloading from a peer
Returns:
(dict) {
@@ -2380,9 +2475,9 @@ Usage:
Options:
--amount=&lt;amount&gt; : (float) amount of credit to send
--address=&lt;address&gt; : (str) address to send credits to
--claim_id=&lt;claim_id&gt; : (float) claim_id of the claim to send to tip to
--amount=&lt;amount&gt; : (Decimal) amount of credit to send
--address=&lt;address&gt; : (str) address to send credits to
--claim_id=&lt;claim_id&gt; : (str) claim_id of the claim to send to tip to
Returns:
If sending to an address:
@@ -2482,7 +2577,7 @@ Returns:
<script src="../assets/javascripts/application.8eb9be28.js"></script>
<script>app.initialize({version:"0.17.2",url:{base:".."}})</script>
<script>app.initialize({version:"0.17.3",url:{base:".."}})</script>
+176 -91
View File
@@ -32,7 +32,7 @@
<meta name="lang:search.tokenizer" content="[\s\-]+">
<link rel="shortcut icon" href="./assets/images/favicon.png">
<meta name="generator" content="mkdocs-0.17.2, mkdocs-material-2.7.0">
<meta name="generator" content="mkdocs-0.17.3, mkdocs-material-2.7.0">
@@ -295,6 +295,13 @@
blob_list
</a>
</li>
<li class="md-nav__item">
<a href="#blob_reflect" title="blob_reflect" class="md-nav__link">
blob_reflect
</a>
</li>
<li class="md-nav__item">
@@ -463,6 +470,13 @@
peer_list
</a>
</li>
<li class="md-nav__item">
<a href="#peer_ping" title="peer_ping" class="md-nav__link">
peer_ping
</a>
</li>
<li class="md-nav__item">
@@ -470,6 +484,13 @@
publish
</a>
</li>
<li class="md-nav__item">
<a href="#report_bug" title="report_bug" class="md-nav__link">
report_bug
</a>
</li>
<li class="md-nav__item">
@@ -709,6 +730,13 @@
blob_list
</a>
</li>
<li class="md-nav__item">
<a href="#blob_reflect" title="blob_reflect" class="md-nav__link">
blob_reflect
</a>
</li>
<li class="md-nav__item">
@@ -877,6 +905,13 @@
peer_list
</a>
</li>
<li class="md-nav__item">
<a href="#peer_ping" title="peer_ping" class="md-nav__link">
peer_ping
</a>
</li>
<li class="md-nav__item">
@@ -884,6 +919,13 @@
publish
</a>
</li>
<li class="md-nav__item">
<a href="#report_bug" title="report_bug" class="md-nav__link">
report_bug
</a>
</li>
<li class="md-nav__item">
@@ -1071,12 +1113,11 @@
<pre><code class="text">Announce blobs to the DHT
Args:
'announce_all' : (bool) announce all the blobs possessed by user
'blob_hash' : (str) announce a blob, specified by blob_hash
'stream_hash' : (str) announce all blobs associated with
stream_hash
'sd_hash' : (str) announce all blobs associated with
sd_hash and the sd_hash itself
'blob_hash' : (str) announce a blob, specified by blob_hash
'stream_hash' : (str) announce all blobs associated with
stream_hash
'sd_hash' : (str) announce all blobs associated with
sd_hash and the sd_hash itself
Returns:
(bool) true if successful
@@ -1144,6 +1185,16 @@ Returns:
(list) List of blob hashes
</code></pre>
<h2 id="blob_reflect">blob_reflect</h2>
<pre><code class="text">Reflects specified blobs
Args:
'reflector_server' : (str) reflector address
Returns:
(list) reflected blob hashes
</code></pre>
<h2 id="blob_reflect_all">blob_reflect_all</h2>
<pre><code class="text">Reflects all saved blobs
@@ -1179,7 +1230,7 @@ Returns:
<pre><code class="text">Import serialized channel signing information (to allow signing new claims to the channel)
Args:
'serialized_certificate_info' : (str) certificate info
'serialized_certificate_info' (required) : (str) certificate info
Returns:
(dict) Result dictionary
@@ -1329,7 +1380,7 @@ Returns:
'is_spent': (bool) true if claim is abandoned, false otherwise
'name': (str) name of the claim
'permanent_url': (str) permanent url of the claim,
'txid': (str) txid of the cliam
'txid': (str) txid of the claim
'nout': (int) nout of the claim
'value': (str) value of the claim
},
@@ -1428,12 +1479,12 @@ Returns:
<h2 id="cli_test_command">cli_test_command</h2>
<pre><code class="text">This command is only for testing the CLI argument parsing
Args:
'a_arg' : a arg
'b_arg' : b arg
'pos_arg' (required) : pos arg
'pos_args' : pos args
'pos_arg2' : pos arg 2
'pos_arg3' : pos arg 3
'a_arg' : (bool) a arg
'b_arg' : (bool) b arg
'pos_arg' (required) : (int) pos arg
'pos_args' : (int) pos args
'pos_arg2' : (int) pos arg 2
'pos_arg3' : (int) pos arg 3
Returns:
pos args
@@ -1501,6 +1552,9 @@ Args:
'claim_name' : (str) get file with matching claim name
'full_status' : (bool) full status, populate the
'message' and 'size' fields
'sort' : (str) sort by any property, like 'file_name'
or 'metadata.author'; to specify direction
append ',asc' or ',desc'
Returns:
(list) List of files
@@ -1624,7 +1678,19 @@ Args:
'timeout' : (int) peer search timeout in seconds
Returns:
(list) List of contacts
(list) List of contact dictionaries {'host': &lt;peer ip&gt;, 'port': &lt;peer port&gt;, 'node_id': &lt;peer node id&gt;}
</code></pre>
<h2 id="peer_ping">peer_ping</h2>
<pre><code class="text">Send a kademlia ping to the specified peer. If address and port are provided the peer is directly pinged,
if not provided the peer is located first.
Args:
'address' : (str) ip address of the peer
'port' : (int) udp port of the peer
Returns:
(str) pong, or {'error': &lt;error message&gt;} if an error is encountered
</code></pre>
<h2 id="publish">publish</h2>
@@ -1644,40 +1710,40 @@ fee, title, description, author, language, license, license_url, thumbnail, prev
or sources. Individual arguments will overwrite the fields specified in metadata argument.
Args:
'name' (required) : (str) name of the content
'bid' (required) : (float) amount to back the claim
'metadata' : (dict) ClaimDict to associate with the claim.
'file_path' : (str) path to file to be associated with name. If provided,
a lbry stream of this file will be used in 'sources'.
If no path is given but a sources dict is provided,
it will be used. If neither are provided, an
error is raised.
'fee' : (dict) Dictionary representing key fee to download content:
{
'currency': currency_symbol,
'amount': float,
'address': str, optional
}
supported currencies: LBC, USD, BTC
If an address is not provided a new one will be
automatically generated. Default fee is zero.
'title' : (str) title of the publication
'description' : (str) description of the publication
'author' : (str) author of the publication
'language' : (str) language of the publication
'license' : (str) publication license
'license_url' : (str) publication license url
'thumbnail' : (str) thumbnail url
'preview' : (str) preview url
'nsfw' : (bool) title of the publication
'sources' : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file
'channel_name' : (str) name of the publisher channel name in the wallet
'channel_id' : (str) claim id of the publisher channel, does not check
for channel claim being in the wallet. This allows
publishing to a channel where only the certificate
private key is in the wallet.
'claim_address' : (str) address where the claim is sent to, if not specified
new address wil automatically be created
'name' (required) : (str) name of the content
'bid' (required) : (decimal) amount to back the claim
'metadata' : (dict) ClaimDict to associate with the claim.
'file_path' : (str) path to file to be associated with name. If provided,
a lbry stream of this file will be used in 'sources'.
If no path is given but a sources dict is provided,
it will be used. If neither are provided, an
error is raised.
'fee' : (dict) Dictionary representing key fee to download content:
{
'currency': currency_symbol,
'amount': decimal,
'address': str, optional
}
supported currencies: LBC, USD, BTC
If an address is not provided a new one will be
automatically generated. Default fee is zero.
'title' : (str) title of the publication
'description' : (str) description of the publication
'author' : (str) author of the publication
'language' : (str) language of the publication
'license' : (str) publication license
'license_url' : (str) publication license url
'thumbnail' : (str) thumbnail url
'preview' : (str) preview url
'nsfw' : (bool) whether the content is nsfw
'sources' : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file
'channel_name' : (str) name of the publisher channel name in the wallet
'channel_id' : (str) claim id of the publisher channel, does not check
for channel claim being in the wallet. This allows
publishing to a channel where only the certificate
private key is in the wallet.
'claim_address' : (str) address where the claim is sent to, if not specified
new address will automatically be created
Returns:
(dict) Dictionary containing result of the claim
@@ -1685,11 +1751,21 @@ Returns:
'tx' : (str) hex encoded transaction
'txid' : (str) txid of resulting claim
'nout' : (int) nout of the resulting claim
'fee' : (float) fee paid for the claim transaction
'fee' : (decimal) fee paid for the claim transaction
'claim_id' : (str) claim ID of the resulting claim
}
</code></pre>
<h2 id="report_bug">report_bug</h2>
<pre><code class="text">Report a bug to slack
Args:
'message' (required) : (str) Description of the bug
Returns:
(bool) true if successful
</code></pre>
<h2 id="resolve">resolve</h2>
<pre><code class="text">Resolve given LBRY URIs
@@ -1779,6 +1855,7 @@ Returns:
&lt;bucket index&gt;: [
{
&quot;address&quot;: (str) peer address,
&quot;port&quot;: (int) peer udp port
&quot;node_id&quot;: (str) peer node id,
&quot;blobs&quot;: (list) blob hashes announced by peer
}
@@ -1838,50 +1915,58 @@ Returns:
<pre><code class="text">Get daemon status
Args:
'session_status' : (bool) include session status in results
'dht_status' : (bool) include dht network and peer status
None
Returns:
(dict) lbrynet-daemon status
{
'lbry_id': lbry peer id, base58,
'installation_id': installation id, base58,
'is_running': bool,
'installation_id': (str) installation id - base58,
'is_running': (bool),
'is_first_run': bool,
'startup_status': {
'code': status code,
'message': status message
'skipped_components': (list) [names of skipped components (str)],
'startup_status': { Does not include components which have been skipped
'database': (bool),
'wallet': (bool),
'session': (bool),
'dht': (bool),
'hash_announcer': (bool),
'stream_identifier': (bool),
'file_manager': (bool),
'blob_manager': (bool),
'blockchain_headers': (bool),
'peer_protocol_server': (bool),
'reflector': (bool),
'upnp': (bool),
'exchange_rate_manager': (bool),
},
'connection_status': {
'code': connection status code,
'message': connection status message
'code': (str) connection status code,
'message': (str) connection status message
},
'blockchain_status': {
'blocks': local blockchain height,
'blocks_behind': remote_height - local_height,
'best_blockhash': block hash of most recent block,
'blockchain_headers': {
'downloading_headers': (bool),
'download_progress': (float) 0-100.0
},
'wallet_is_encrypted': bool,
If given the session status option:
'session_status': {
'managed_blobs': count of blobs in the blob manager,
'managed_streams': count of streams in the file manager
'announce_queue_size': number of blobs currently queued to be announced
'should_announce_blobs': number of blobs that should be announced
}
If given the dht status option:
'dht_status': {
'kbps_received': current kbps receiving,
'kbps_sent': current kdps being sent,
'total_bytes_sent': total bytes sent,
'total_bytes_received': total bytes received,
'queries_received': number of queries received per second,
'queries_sent': number of queries sent per second,
'recent_contacts': count of recently contacted peers,
'unique_contacts': count of unique peers
},
'wallet': {
'blocks': (int) local blockchain height,
'blocks_behind': (int) remote_height - local_height,
'best_blockhash': (str) block hash of most recent block,
'is_encrypted': (bool),
'is_locked': (bool),
},
'dht': {
'node_id': (str) lbry dht node id - hex encoded,
'peers_in_routing_table': (int) the number of peers in the routing table,
},
'blob_manager': {
'finished_blobs': (int) number of finished blobs in the blob manager,
},
'hash_announcer': {
'announce_queue_size': (int) number of blobs currently queued to be announced
},
'file_manager': {
'managed_files': (int) count of files in the file manager,
}
}
</code></pre>
@@ -1892,7 +1977,7 @@ Args:
'uri' (required) : (str) check availability for this uri
'search_timeout' : (int) how long to search for peers for the blob
in the dht
'search_timeout' : (int) how long to try downloading from a peer
'blob_timeout' : (int) how long to try downloading from a peer
Returns:
(dict) {
@@ -2124,9 +2209,9 @@ to the owner of a claim specified by uri. A tip is a claim support where the rec
of the support is the claim address for the claim being supported.
Args:
'amount' (required) : (float) amount of credit to send
'address' (required) : (str) address to send credits to
'claim_id' (required) : (float) claim_id of the claim to send to tip to
'amount' (required) : (Decimal) amount of credit to send
'address' (required) : (str) address to send credits to
'claim_id' (required) : (str) claim_id of the claim to send to tip to
Returns:
If sending to an address:
@@ -2218,7 +2303,7 @@ Returns:
<script src="./assets/javascripts/application.8eb9be28.js"></script>
<script>app.initialize({version:"0.17.2",url:{base:"."}})</script>
<script>app.initialize({version:"0.17.3",url:{base:"."}})</script>
+54 -24
View File
@@ -2,7 +2,7 @@
"docs": [
{
"location": "/",
"text": "LBRY JSON-RPC API Documentation\n\n\nblob_announce\n\n\nAnnounce blobs to the DHT\n\nArgs:\n 'announce_all' : (bool) announce all the blobs possessed by user\n 'blob_hash' : (str) announce a blob, specified by blob_hash\n 'stream_hash' : (str) announce all blobs associated with\n stream_hash\n 'sd_hash' : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful\n\n\n\n\nblob_availability\n\n\nGet blob availability\n\nArgs:\n 'blob_hash' : (str) check availability for this blob hash\n 'search_timeout' : (int) how long to search for peers for the blob\n in the dht\n 'blob_timeout' : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n \nis_available\n: \nbool, true if blob is available from a peer from peer list\n\n \nreachable_peers\n: [\nip\n:\nport\n],\n \nunreachable_peers\n: [\nip\n:\nport\n]\n }\n\n\n\n\nblob_delete\n\n\nDelete a blob\n\nArgs:\n 'blob_hash' : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message\n\n\n\n\nblob_get\n\n\nDownload and return a blob\n\nArgs:\n 'blob_hash' (required) : (str) blob hash of the blob to get\n 'timeout' : (int) timeout in number of seconds\n 'encoding' : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n 'payment_rate_manager' : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data\n\n\n\n\nblob_list\n\n\nReturns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nArgs:\n 'needed' : (bool) only return needed blobs\n 'finished' : (bool) only return finished blobs\n 'uri' : (str) filter blobs by stream in a uri\n 'stream_hash' : (str) filter blobs by stream hash\n 'sd_hash' : (str) filter blobs by sd hash\n 'page_size' : (int) results page size\n 'page' : (int) page of results to return\n\nReturns:\n (list) List of blob hashes\n\n\n\n\nblob_reflect_all\n\n\nReflects all saved blobs\n\nArgs:\n None\n\nReturns:\n (bool) true if successful\n\n\n\n\nblock_show\n\n\nGet contents of a block\n\nArgs:\n 'blockhash' (required) : (str) hash of the block to look up\n 'height' (required) : (int) height of the block to look up\n\nReturns:\n (dict) Requested block\n\n\n\n\nchannel_export\n\n\nExport serialized channel signing information for a given certificate claim id\n\nArgs:\n 'claim_id' (required) : (str) Claim ID to export information about\n\nReturns:\n (str) Serialized certificate information\n\n\n\n\nchannel_import\n\n\nImport serialized channel signing information (to allow signing new claims to the channel)\n\nArgs:\n 'serialized_certificate_info' : (str) certificate info\n\nReturns:\n (dict) Result dictionary\n\n\n\n\nchannel_list\n\n\nGet certificate claim infos for channels that can be published to\n\nArgs:\n None\n\nReturns:\n (list) ClaimDict, includes 'is_mine' field to indicate if the certificate claim\n is in the wallet.\n\n\n\n\nchannel_new\n\n\nGenerate a publisher key and create a new '@' prefixed certificate claim\n\nArgs:\n 'channel_name' (required) : (str) name of the channel prefixed with '@'\n 'amount' (required) : (float) bid amount on the channel\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }\n\n\n\n\nclaim_abandon\n\n\nAbandon a name and reclaim credits from the claim\n\nArgs:\n 'claim_id' : (str) claim_id of the claim to abandon\n 'txid' : (str) txid of the claim to abandon\n 'nout' : (int) nout of the claim to abandon\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n txid : (str) txid of resulting transaction\n fee : (float) fee paid for the transaction\n }\n\n\n\n\nclaim_list\n\n\nList current claims and information about them for a given name\n\nArgs:\n 'name' (required) : (str) name of the claim to list info about\n\nReturns:\n (dict) State of claims assigned for the name\n {\n 'claims': (list) list of claims for the Line truncated
"text": "LBRY JSON-RPC API Documentation\n\n\nblob_announce\n\n\nAnnounce blobs to the DHT\n\nArgs:\n 'blob_hash' : (str) announce a blob, specified by blob_hash\n 'stream_hash' : (str) announce all blobs associated with\n stream_hash\n 'sd_hash' : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful\n\n\n\n\nblob_availability\n\n\nGet blob availability\n\nArgs:\n 'blob_hash' : (str) check availability for this blob hash\n 'search_timeout' : (int) how long to search for peers for the blob\n in the dht\n 'blob_timeout' : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n \nis_available\n: \nbool, true if blob is available from a peer from peer list\n\n \nreachable_peers\n: [\nip\n:\nport\n],\n \nunreachable_peers\n: [\nip\n:\nport\n]\n }\n\n\n\n\nblob_delete\n\n\nDelete a blob\n\nArgs:\n 'blob_hash' : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message\n\n\n\n\nblob_get\n\n\nDownload and return a blob\n\nArgs:\n 'blob_hash' (required) : (str) blob hash of the blob to get\n 'timeout' : (int) timeout in number of seconds\n 'encoding' : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n 'payment_rate_manager' : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data\n\n\n\n\nblob_list\n\n\nReturns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nArgs:\n 'needed' : (bool) only return needed blobs\n 'finished' : (bool) only return finished blobs\n 'uri' : (str) filter blobs by stream in a uri\n 'stream_hash' : (str) filter blobs by stream hash\n 'sd_hash' : (str) filter blobs by sd hash\n 'page_size' : (int) results page size\n 'page' : (int) page of results to return\n\nReturns:\n (list) List of blob hashes\n\n\n\n\nblob_reflect\n\n\nReflects specified blobs\n\nArgs:\n 'reflector_server' : (str) reflector address\n\nReturns:\n (list) reflected blob hashes\n\n\n\n\nblob_reflect_all\n\n\nReflects all saved blobs\n\nArgs:\n None\n\nReturns:\n (bool) true if successful\n\n\n\n\nblock_show\n\n\nGet contents of a block\n\nArgs:\n 'blockhash' (required) : (str) hash of the block to look up\n 'height' (required) : (int) height of the block to look up\n\nReturns:\n (dict) Requested block\n\n\n\n\nchannel_export\n\n\nExport serialized channel signing information for a given certificate claim id\n\nArgs:\n 'claim_id' (required) : (str) Claim ID to export information about\n\nReturns:\n (str) Serialized certificate information\n\n\n\n\nchannel_import\n\n\nImport serialized channel signing information (to allow signing new claims to the channel)\n\nArgs:\n 'serialized_certificate_info' (required) : (str) certificate info\n\nReturns:\n (dict) Result dictionary\n\n\n\n\nchannel_list\n\n\nGet certificate claim infos for channels that can be published to\n\nArgs:\n None\n\nReturns:\n (list) ClaimDict, includes 'is_mine' field to indicate if the certificate claim\n is in the wallet.\n\n\n\n\nchannel_new\n\n\nGenerate a publisher key and create a new '@' prefixed certificate claim\n\nArgs:\n 'channel_name' (required) : (str) name of the channel prefixed with '@'\n 'amount' (required) : (float) bid amount on the channel\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }\n\n\n\n\nclaim_abandon\n\n\nAbandon a name and reclaim credits from the claim\n\nArgs:\n 'claim_id' : (str) claim_id of the claim to abandon\n 'txid' : (str) txid of the claim to abandon\n 'nout' : (int) nout of the claim to abandon\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n txid : (str) txid of resulting transaction\n fee : (float) fee paid for the transaction\n }\n\n\n\n\nclaim_list\n\n\nList current claims and information about them for a given name\n\nArgs:\n 'name' (required) : (str) name of the claim to list info about\n\nReturns:\n (dict) State Line truncated
"title": "API"
},
{
@@ -12,7 +12,7 @@
},
{
"location": "/#blob_announce",
"text": "Announce blobs to the DHT\n\nArgs:\n 'announce_all' : (bool) announce all the blobs possessed by user\n 'blob_hash' : (str) announce a blob, specified by blob_hash\n 'stream_hash' : (str) announce all blobs associated with\n stream_hash\n 'sd_hash' : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful",
"text": "Announce blobs to the DHT\n\nArgs:\n 'blob_hash' : (str) announce a blob, specified by blob_hash\n 'stream_hash' : (str) announce all blobs associated with\n stream_hash\n 'sd_hash' : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful",
"title": "blob_announce"
},
{
@@ -35,6 +35,11 @@
"text": "Returns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nArgs:\n 'needed' : (bool) only return needed blobs\n 'finished' : (bool) only return finished blobs\n 'uri' : (str) filter blobs by stream in a uri\n 'stream_hash' : (str) filter blobs by stream hash\n 'sd_hash' : (str) filter blobs by sd hash\n 'page_size' : (int) results page size\n 'page' : (int) page of results to return\n\nReturns:\n (list) List of blob hashes",
"title": "blob_list"
},
{
"location": "/#blob_reflect",
"text": "Reflects specified blobs\n\nArgs:\n 'reflector_server' : (str) reflector address\n\nReturns:\n (list) reflected blob hashes",
"title": "blob_reflect"
},
{
"location": "/#blob_reflect_all",
"text": "Reflects all saved blobs\n\nArgs:\n None\n\nReturns:\n (bool) true if successful",
@@ -52,7 +57,7 @@
},
{
"location": "/#channel_import",
"text": "Import serialized channel signing information (to allow signing new claims to the channel)\n\nArgs:\n 'serialized_certificate_info' : (str) certificate info\n\nReturns:\n (dict) Result dictionary",
"text": "Import serialized channel signing information (to allow signing new claims to the channel)\n\nArgs:\n 'serialized_certificate_info' (required) : (str) certificate info\n\nReturns:\n (dict) Result dictionary",
"title": "channel_import"
},
{
@@ -82,7 +87,7 @@
},
{
"location": "/#claim_list_mine",
"text": "List my name claims\n\nArgs:\n None\n\nReturns:\n (list) List of name claims owned by user\n [\n {\n 'address': (str) address that owns the claim\n 'amount': (float) amount assigned to the claim\n 'blocks_to_expiration': (int) number of blocks until it expires\n 'category': (str) claim , update , or support \n 'claim_id': (str) claim ID of the claim\n 'confirmations': (int) number of blocks of confirmations for the claim\n 'expiration_height': (int) the block height which the claim will expire\n 'expired': (bool) true if expired, false otherwise\n 'height': (int) height of the block containing the claim\n 'is_spent': (bool) true if claim is abandoned, false otherwise\n 'name': (str) name of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'txid': (str) txid of the cliam\n 'nout': (int) nout of the claim\n 'value': (str) value of the claim\n },\n ]",
"text": "List my name claims\n\nArgs:\n None\n\nReturns:\n (list) List of name claims owned by user\n [\n {\n 'address': (str) address that owns the claim\n 'amount': (float) amount assigned to the claim\n 'blocks_to_expiration': (int) number of blocks until it expires\n 'category': (str) claim , update , or support \n 'claim_id': (str) claim ID of the claim\n 'confirmations': (int) number of blocks of confirmations for the claim\n 'expiration_height': (int) the block height which the claim will expire\n 'expired': (bool) true if expired, false otherwise\n 'height': (int) height of the block containing the claim\n 'is_spent': (bool) true if claim is abandoned, false otherwise\n 'name': (str) name of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'txid': (str) txid of the claim\n 'nout': (int) nout of the claim\n 'value': (str) value of the claim\n },\n ]",
"title": "claim_list_mine"
},
{
@@ -107,7 +112,7 @@
},
{
"location": "/#cli_test_command",
"text": "This command is only for testing the CLI argument parsing\nArgs:\n 'a_arg' : a arg\n 'b_arg' : b arg\n 'pos_arg' (required) : pos arg\n 'pos_args' : pos args\n 'pos_arg2' : pos arg 2\n 'pos_arg3' : pos arg 3\n\nReturns:\n pos args",
"text": "This command is only for testing the CLI argument parsing\nArgs:\n 'a_arg' : (bool) a arg\n 'b_arg' : (bool) b arg\n 'pos_arg' (required) : (int) pos arg\n 'pos_args' : (int) pos args\n 'pos_arg2' : (int) pos arg 2\n 'pos_arg3' : (int) pos arg 3\n\nReturns:\n pos args",
"title": "cli_test_command"
},
{
@@ -127,7 +132,7 @@
},
{
"location": "/#file_list",
"text": "List files limited by optional filters\n\nArgs:\n 'sd_hash' : (str) get file with matching sd hash\n 'file_name' : (str) get file with matching file name in the\n downloads folder\n 'stream_hash' : (str) get file with matching stream hash\n 'rowid' : (int) get file with matching row id\n 'claim_id' : (str) get file with matching claim id\n 'outpoint' : (str) get file with matching claim outpoint\n 'txid' : (str) get file with matching claim txid\n 'nout' : (int) get file with matching claim nout\n 'channel_claim_id' : (str) get file with matching channel claim id\n 'channel_name' : (str) get file with matching channel name\n 'claim_name' : (str) get file with matching claim name\n 'full_status' : (bool) full status, populate the\n 'message' and 'size' fields\n\nReturns:\n (list) List of files\n\n [\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) None if full_status is false or if claim is not found,\n 'outpoint': (str) None if full_status is false or if claim is not found,\n 'txid': (str) None if full_status is false or if claim is not found,\n 'nout': (int) None if full_status is false or if claim is not found,\n 'metadata': (dict) None if full_status is false or if claim is not found,\n 'channel_claim_id': (str) None if full_status is false or if claim is not found or signed,\n 'channel_name': (str) None if full_status is false or if claim is not found or signed,\n 'claim_name': (str) None if full_status is false or if claim is not found\n },\n ]",
"text": "List files limited by optional filters\n\nArgs:\n 'sd_hash' : (str) get file with matching sd hash\n 'file_name' : (str) get file with matching file name in the\n downloads folder\n 'stream_hash' : (str) get file with matching stream hash\n 'rowid' : (int) get file with matching row id\n 'claim_id' : (str) get file with matching claim id\n 'outpoint' : (str) get file with matching claim outpoint\n 'txid' : (str) get file with matching claim txid\n 'nout' : (int) get file with matching claim nout\n 'channel_claim_id' : (str) get file with matching channel claim id\n 'channel_name' : (str) get file with matching channel name\n 'claim_name' : (str) get file with matching claim name\n 'full_status' : (bool) full status, populate the\n 'message' and 'size' fields\n 'sort' : (str) sort by any property, like 'file_name'\n or 'metadata.author'; to specify direction\n append ',asc' or ',desc'\n\nReturns:\n (list) List of files\n\n [\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) None if full_status is false or if claim is not found,\n 'outpoint': (str) None if full_status is false or if claim is not found,\n 'txid': (str) None if full_status is false or if claim is not found,\n 'nout': (int) None if full_status is false or if claim is not found,\n 'metadata': (dict) None if full_status is false or if claim is not found,\n 'channel_claim_id': (str) None if full_status is false or if claim is not found or signed,\n 'channel_name': (str) None if full_status is false or if claim is not found or signed,\n 'claim_name': (str) None if full_status is false or if claim is not found\n },\n ]",
"title": "file_list"
},
{
@@ -152,14 +157,24 @@
},
{
"location": "/#peer_list",
"text": "Get peers for blob hash\n\nArgs:\n 'blob_hash' (required) : (str) find available peers for this blob hash\n 'timeout' : (int) peer search timeout in seconds\n\nReturns:\n (list) List of contacts",
"text": "Get peers for blob hash\n\nArgs:\n 'blob_hash' (required) : (str) find available peers for this blob hash\n 'timeout' : (int) peer search timeout in seconds\n\nReturns:\n (list) List of contact dictionaries {'host': peer ip , 'port': peer port , 'node_id': peer node id }",
"title": "peer_list"
},
{
"location": "/#peer_ping",
"text": "Send a kademlia ping to the specified peer. If address and port are provided the peer is directly pinged,\nif not provided the peer is located first.\n\nArgs:\n 'address' : (str) ip address of the peer\n 'port' : (int) udp port of the peer\n\nReturns:\n (str) pong, or {'error': error message } if an error is encountered",
"title": "peer_ping"
},
{
"location": "/#publish",
"text": "Make a new name claim and publish associated data to lbrynet,\nupdate over existing claim if user already has a claim for name.\n\nFields required in the final Metadata are:\n 'title'\n 'description'\n 'author'\n 'language'\n 'license'\n 'nsfw'\n\nMetadata can be set by either using the metadata argument or by setting individual arguments\nfee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\nor sources. Individual arguments will overwrite the fields specified in metadata argument.\n\nArgs:\n 'name' (required) : (str) name of the content\n 'bid' (required) : (float) amount to back the claim\n 'metadata' : (dict) ClaimDict to associate with the claim.\n 'file_path' : (str) path to file to be associated with name. If provided,\n a lbry stream of this file will be used in 'sources'.\n If no path is given but a sources dict is provided,\n it will be used. If neither are provided, an\n error is raised.\n 'fee' : (dict) Dictionary representing key fee to download content:\n {\n 'currency': currency_symbol,\n 'amount': float,\n 'address': str, optional\n }\n supported currencies: LBC, USD, BTC\n If an address is not provided a new one will be\n automatically generated. Default fee is zero.\n 'title' : (str) title of the publication\n 'description' : (str) description of the publication\n 'author' : (str) author of the publication\n 'language' : (str) language of the publication\n 'license' : (str) publication license\n 'license_url' : (str) publication license url\n 'thumbnail' : (str) thumbnail url\n 'preview' : (str) preview url\n 'nsfw' : (bool) title of the publication\n 'sources' : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file\n 'channel_name' : (str) name of the publisher channel name in the wallet\n 'channel_id' : (str) claim id of the publisher channel, does not check\n for channel claim being in the wallet. This allows\n publishing to a channel where only the certificate\n private key is in the wallet.\n 'claim_address' : (str) address where the claim is sent to, if not specified\n new address wil automatically be created\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"text": "Make a new name claim and publish associated data to lbrynet,\nupdate over existing claim if user already has a claim for name.\n\nFields required in the final Metadata are:\n 'title'\n 'description'\n 'author'\n 'language'\n 'license'\n 'nsfw'\n\nMetadata can be set by either using the metadata argument or by setting individual arguments\nfee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\nor sources. Individual arguments will overwrite the fields specified in metadata argument.\n\nArgs:\n 'name' (required) : (str) name of the content\n 'bid' (required) : (decimal) amount to back the claim\n 'metadata' : (dict) ClaimDict to associate with the claim.\n 'file_path' : (str) path to file to be associated with name. If provided,\n a lbry stream of this file will be used in 'sources'.\n If no path is given but a sources dict is provided,\n it will be used. If neither are provided, an\n error is raised.\n 'fee' : (dict) Dictionary representing key fee to download content:\n {\n 'currency': currency_symbol,\n 'amount': decimal,\n 'address': str, optional\n }\n supported currencies: LBC, USD, BTC\n If an address is not provided a new one will be\n automatically generated. Default fee is zero.\n 'title' : (str) title of the publication\n 'description' : (str) description of the publication\n 'author' : (str) author of the publication\n 'language' : (str) language of the publication\n 'license' : (str) publication license\n 'license_url' : (str) publication license url\n 'thumbnail' : (str) thumbnail url\n 'preview' : (str) preview url\n 'nsfw' : (bool) whether the content is nsfw\n 'sources' : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file\n 'channel_name' : (str) name of the publisher channel name in the wallet\n 'channel_id' : (str) claim id of the publisher channel, does not check\n for channel claim being in the wallet. This allows\n publishing to a channel where only the certificate\n private key is in the wallet.\n 'claim_address' : (str) address where the claim is sent to, if not specified\n new address will automatically be created\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (decimal) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "publish"
},
{
"location": "/#report_bug",
"text": "Report a bug to slack\n\nArgs:\n 'message' (required) : (str) Description of the bug\n\nReturns:\n (bool) true if successful",
"title": "report_bug"
},
{
"location": "/#resolve",
"text": "Resolve given LBRY URIs\n\nArgs:\n 'force' : (bool) force refresh and ignore cache\n 'uri' (required) : (str) uri to resolve\n 'uris' : (list) uris to resolve\n\nReturns:\n Dictionary of results, keyed by uri\n ' uri ': {\n If a resolution error occurs:\n 'error': Error message\n\n If the uri resolves to a channel or a claim in a channel:\n 'certificate': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the certificate claim,\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}],\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n\n If the uri resolves to a channel:\n 'claims_in_channel': (int) number of claims in the channel,\n\n If the uri resolves to a claim:\n 'claim': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the claim,\n 'channel_name': (str) channel name if claim is in a channel\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}]\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n }",
@@ -172,7 +187,7 @@
},
{
"location": "/#routing_table_get",
"text": "Get DHT routing information\n\nArgs:\n None\n\nReturns:\n (dict) dictionary containing routing and contact information\n {\n buckets : {\n bucket index : [\n {\n address : (str) peer address,\n node_id : (str) peer node id,\n blobs : (list) blob hashes announced by peer\n }\n ]\n },\n contacts : (list) contact node ids,\n blob_hashes : (list) all of the blob hashes stored by peers in the list of buckets,\n node_id : (str) the local dht node id\n }",
"text": "Get DHT routing information\n\nArgs:\n None\n\nReturns:\n (dict) dictionary containing routing and contact information\n {\n buckets : {\n bucket index : [\n {\n address : (str) peer address,\n port : (int) peer udp port\n node_id : (str) peer node id,\n blobs : (list) blob hashes announced by peer\n }\n ]\n },\n contacts : (list) contact node ids,\n blob_hashes : (list) all of the blob hashes stored by peers in the list of buckets,\n node_id : (str) the local dht node id\n }",
"title": "routing_table_get"
},
{
@@ -187,12 +202,12 @@
},
{
"location": "/#status",
"text": "Get daemon status\n\nArgs:\n 'session_status' : (bool) include session status in results\n 'dht_status' : (bool) include dht network and peer status\n\nReturns:\n (dict) lbrynet-daemon status\n {\n 'lbry_id': lbry peer id, base58,\n 'installation_id': installation id, base58,\n 'is_running': bool,\n 'is_first_run': bool,\n 'startup_status': {\n 'code': status code,\n 'message': status message\n },\n 'connection_status': {\n 'code': connection status code,\n 'message': connection status message\n },\n 'blockchain_status': {\n 'blocks': local blockchain height,\n 'blocks_behind': remote_height - local_height,\n 'best_blockhash': block hash of most recent block,\n },\n 'wallet_is_encrypted': bool,\n\n If given the session status option:\n 'session_status': {\n 'managed_blobs': count of blobs in the blob manager,\n 'managed_streams': count of streams in the file manager\n 'announce_queue_size': number of blobs currently queued to be announced\n 'should_announce_blobs': number of blobs that should be announced\n }\n\n If given the dht status option:\n 'dht_status': {\n 'kbps_received': current kbps receiving,\n 'kbps_sent': current kdps being sent,\n 'total_bytes_sent': total bytes sent,\n 'total_bytes_received': total bytes received,\n 'queries_received': number of queries received per second,\n 'queries_sent': number of queries sent per second,\n 'recent_contacts': count of recently contacted peers,\n 'unique_contacts': count of unique peers\n },\n }",
"text": "Get daemon status\n\nArgs:\n None\n\nReturns:\n (dict) lbrynet-daemon status\n {\n 'installation_id': (str) installation id - base58,\n 'is_running': (bool),\n 'is_first_run': bool,\n 'skipped_components': (list) [names of skipped components (str)],\n 'startup_status': { Does not include components which have been skipped\n 'database': (bool),\n 'wallet': (bool),\n 'session': (bool),\n 'dht': (bool),\n 'hash_announcer': (bool),\n 'stream_identifier': (bool),\n 'file_manager': (bool),\n 'blob_manager': (bool),\n 'blockchain_headers': (bool),\n 'peer_protocol_server': (bool),\n 'reflector': (bool),\n 'upnp': (bool),\n 'exchange_rate_manager': (bool),\n },\n 'connection_status': {\n 'code': (str) connection status code,\n 'message': (str) connection status message\n },\n 'blockchain_headers': {\n 'downloading_headers': (bool),\n 'download_progress': (float) 0-100.0\n },\n 'wallet': {\n 'blocks': (int) local blockchain height,\n 'blocks_behind': (int) remote_height - local_height,\n 'best_blockhash': (str) block hash of most recent block,\n 'is_encrypted': (bool),\n 'is_locked': (bool),\n },\n 'dht': {\n 'node_id': (str) lbry dht node id - hex encoded,\n 'peers_in_routing_table': (int) the number of peers in the routing table,\n },\n 'blob_manager': {\n 'finished_blobs': (int) number of finished blobs in the blob manager,\n },\n 'hash_announcer': {\n 'announce_queue_size': (int) number of blobs currently queued to be announced\n },\n 'file_manager': {\n 'managed_files': (int) count of files in the file manager,\n }\n }",
"title": "status"
},
{
"location": "/#stream_availability",
"text": "Get stream availability for lbry uri\n\nArgs:\n 'uri' (required) : (str) check availability for this uri\n 'search_timeout' : (int) how long to search for peers for the blob\n in the dht\n 'search_timeout' : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n 'is_available': bool ,\n 'did_decode': bool ,\n 'did_resolve': bool ,\n 'is_stream': bool ,\n 'num_blobs_in_stream': int ,\n 'sd_hash': str ,\n 'sd_blob_availability': dict see `blob_availability`,\n 'head_blob_hash': str ,\n 'head_blob_availability': dict see `blob_availability`,\n 'use_upnp': bool ,\n 'upnp_redirect_is_set': bool ,\n 'error': None | str error message\n }",
"text": "Get stream availability for lbry uri\n\nArgs:\n 'uri' (required) : (str) check availability for this uri\n 'search_timeout' : (int) how long to search for peers for the blob\n in the dht\n 'blob_timeout' : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n 'is_available': bool ,\n 'did_decode': bool ,\n 'did_resolve': bool ,\n 'is_stream': bool ,\n 'num_blobs_in_stream': int ,\n 'sd_hash': str ,\n 'sd_blob_availability': dict see `blob_availability`,\n 'head_blob_hash': str ,\n 'head_blob_availability': dict see `blob_availability`,\n 'use_upnp': bool ,\n 'upnp_redirect_is_set': bool ,\n 'error': None | str error message\n }",
"title": "stream_availability"
},
{
@@ -262,7 +277,7 @@
},
{
"location": "/#wallet_send",
"text": "Send credits. If given an address, send credits to it. If given a claim id, send a tip\nto the owner of a claim specified by uri. A tip is a claim support where the recipient\nof the support is the claim address for the claim being supported.\n\nArgs:\n 'amount' (required) : (float) amount of credit to send\n 'address' (required) : (str) address to send credits to\n 'claim_id' (required) : (float) claim_id of the claim to send to tip to\n\nReturns:\n If sending to an address:\n (bool) true if payment successfully scheduled\n\n If sending a claim tip:\n (dict) Dictionary containing the result of the support\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"text": "Send credits. If given an address, send credits to it. If given a claim id, send a tip\nto the owner of a claim specified by uri. A tip is a claim support where the recipient\nof the support is the claim address for the claim being supported.\n\nArgs:\n 'amount' (required) : (Decimal) amount of credit to send\n 'address' (required) : (str) address to send credits to\n 'claim_id' (required) : (str) claim_id of the claim to send to tip to\n\nReturns:\n If sending to an address:\n (bool) true if payment successfully scheduled\n\n If sending a claim tip:\n (dict) Dictionary containing the result of the support\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"title": "wallet_send"
},
{
@@ -277,7 +292,7 @@
},
{
"location": "/cli/",
"text": "LBRY JSON-RPC API Documentation\n\n\nblob_announce\n\n\nAnnounce blobs to the DHT\n\nUsage:\n blob_announce [--announce_all] [\nblob_hash\n | --blob_hash=\nblob_hash\n]\n [\nstream_hash\n | --stream_hash=\nstream_hash\n]\n [\nsd_hash\n | --sd_hash=\nsd_hash\n]\n\n\nOptions:\n --announce_all=\nannounce_all\n : (bool) announce all the blobs possessed by user\n --blob_hash=\nblob_hash\n : (str) announce a blob, specified by blob_hash\n --stream_hash=\nstream_hash\n : (str) announce all blobs associated with\n stream_hash\n --sd_hash=\nsd_hash\n : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful\n\n\n\n\nblob_availability\n\n\nGet blob availability\n\nUsage:\n blob_availability (\nblob_hash\n) [\nsearch_timeout\n | --search_timeout=\nsearch_timeout\n]\n [\nblob_timeout\n | --blob_timeout=\nblob_timeout\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) check availability for this blob hash\n --search_timeout=\nsearch_timeout\n : (int) how long to search for peers for the blob\n in the dht\n --blob_timeout=\nblob_timeout\n : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n \nis_available\n: \nbool, true if blob is available from a peer from peer list\n\n \nreachable_peers\n: [\nip\n:\nport\n],\n \nunreachable_peers\n: [\nip\n:\nport\n]\n }\n\n\n\n\nblob_delete\n\n\nDelete a blob\n\nUsage:\n blob_delete (\nblob_hash\n | --blob_hash=\nblob_hash)\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message\n\n\n\n\nblob_get\n\n\nDownload and return a blob\n\nUsage:\n blob_get (\nblob_hash\n | --blob_hash=\nblob_hash\n) [--timeout=\ntimeout\n]\n [--encoding=\nencoding\n] [--payment_rate_manager=\npayment_rate_manager\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) blob hash of the blob to get\n --timeout=\ntimeout\n : (int) timeout in number of seconds\n --encoding=\nencoding\n : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n --payment_rate_manager=\npayment_rate_manager\n : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data\n\n\n\n\nblob_list\n\n\nReturns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nUsage:\n blob_list [--needed] [--finished] [\nuri\n | --uri=\nuri\n]\n [\nstream_hash\n | --stream_hash=\nstream_hash\n]\n [\nsd_hash\n | --sd_hash=\nsd_hash\n]\n [\npage_size\n | --page_size=\npage_size\n]\n [\npage\n | --page=\npage\n]\n\n\nOptions:\n --needed : (bool) only return needed blobs\n --finished : (bool) only return finished blobs\n --uri=\nuri\n : (str) filter blobs by stream in a uri\n --stream_hash=\nstream_hash\n : (str) filter blobs by stream hash\n --sd_hash=\nsd_hash\n : (str) filter blobs by sd hash\n --page_size=\npage_size\n : (int) results page size\n --page=\npage\n : (int) page of results to return\n\nReturns:\n (list) List of blob hashes\n\n\n\n\nblob_reflect_all\n\n\nReflects all saved blobs\n\nUsage:\n blob_reflect_all\n\n\nOptions:\n None\n\nReturns:\n (bool) true if successful\n\n\n\n\nblock_show\n\n\nGet contents of a block\n\nUsage:\n block_show (\nblockhash\n | --blockhash=\nblockhash\n) | (\nheight\n | --height=\nheight\n)\n\n\nOptions:\n --blockhash=\nblockhash\n : (str) hash of the block to look up\n --height=\nheight\n : (int) height of the block to look up\n\nReturns:\n (dict) Requested block\n\n\n\n\nchannel_export\n\n\nExport serialized channel signing information for a given certificate claim id\n\nUsage:\n channel_export (\nclaim_id\n | --claim_id=\nclaim_id\n)\n\n\nOptions:\n --claim_id=\nclaim_id\n : (str) Claim ID to export information about\n\nReturns:\n (str) Serialized certificate information\n\n\n\n\nchannel_import\n\n\nImport serialized channel signing informatLine truncated
"text": "LBRY JSON-RPC API Documentation\n\n\nblob_announce\n\n\nAnnounce blobs to the DHT\n\nUsage:\n blob_announce [\nblob_hash\n | --blob_hash=\nblob_hash\n]\n [\nstream_hash\n | --stream_hash=\nstream_hash\n] | [\nsd_hash\n | --sd_hash=\nsd_hash\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) announce a blob, specified by blob_hash\n --stream_hash=\nstream_hash\n : (str) announce all blobs associated with\n stream_hash\n --sd_hash=\nsd_hash\n : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful\n\n\n\n\nblob_availability\n\n\nGet blob availability\n\nUsage:\n blob_availability (\nblob_hash\n) [\nsearch_timeout\n | --search_timeout=\nsearch_timeout\n]\n [\nblob_timeout\n | --blob_timeout=\nblob_timeout\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) check availability for this blob hash\n --search_timeout=\nsearch_timeout\n : (int) how long to search for peers for the blob\n in the dht\n --blob_timeout=\nblob_timeout\n : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n \nis_available\n: \nbool, true if blob is available from a peer from peer list\n\n \nreachable_peers\n: [\nip\n:\nport\n],\n \nunreachable_peers\n: [\nip\n:\nport\n]\n }\n\n\n\n\nblob_delete\n\n\nDelete a blob\n\nUsage:\n blob_delete (\nblob_hash\n | --blob_hash=\nblob_hash)\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message\n\n\n\n\nblob_get\n\n\nDownload and return a blob\n\nUsage:\n blob_get (\nblob_hash\n | --blob_hash=\nblob_hash\n) [--timeout=\ntimeout\n]\n [--encoding=\nencoding\n] [--payment_rate_manager=\npayment_rate_manager\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) blob hash of the blob to get\n --timeout=\ntimeout\n : (int) timeout in number of seconds\n --encoding=\nencoding\n : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n --payment_rate_manager=\npayment_rate_manager\n : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data\n\n\n\n\nblob_list\n\n\nReturns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nUsage:\n blob_list [--needed] [--finished] [\nuri\n | --uri=\nuri\n]\n [\nstream_hash\n | --stream_hash=\nstream_hash\n]\n [\nsd_hash\n | --sd_hash=\nsd_hash\n]\n [\npage_size\n | --page_size=\npage_size\n]\n [\npage\n | --page=\npage\n]\n\n\nOptions:\n --needed : (bool) only return needed blobs\n --finished : (bool) only return finished blobs\n --uri=\nuri\n : (str) filter blobs by stream in a uri\n --stream_hash=\nstream_hash\n : (str) filter blobs by stream hash\n --sd_hash=\nsd_hash\n : (str) filter blobs by sd hash\n --page_size=\npage_size\n : (int) results page size\n --page=\npage\n : (int) page of results to return\n\nReturns:\n (list) List of blob hashes\n\n\n\n\nblob_reflect\n\n\nReflects specified blobs\n\nUsage:\n blob_reflect (\nblob_hashes\n...) [--reflector_server=\nreflector_server\n]\n\n\nOptions:\n --reflector_server=\nreflector_server\n : (str) reflector address\n\nReturns:\n (list) reflected blob hashes\n\n\n\n\nblob_reflect_all\n\n\nReflects all saved blobs\n\nUsage:\n blob_reflect_all\n\n\nOptions:\n None\n\nReturns:\n (bool) true if successful\n\n\n\n\nblock_show\n\n\nGet contents of a block\n\nUsage:\n block_show (\nblockhash\n | --blockhash=\nblockhash\n) | (\nheight\n | --height=\nheight\n)\n\n\nOptions:\n --blockhash=\nblockhash\n : (str) hash of the block to look up\n --height=\nheight\n : (int) height of the block to look up\n\nReturns:\n (dict) Requested block\n\n\n\n\nchannel_export\n\n\nExport serialized channel signing information for a given certificate claim id\n\nUsage:\n channel_export (\nclaim_id\n | --claim_id=\nclaim_id\n)\n\n\nOptions:\n --claim_id=\nclaim_id\n : (str) Claim ID to export informatiLine truncated
"title": "CLI"
},
{
@@ -287,7 +302,7 @@
},
{
"location": "/cli/#blob_announce",
"text": "Announce blobs to the DHT\n\nUsage:\n blob_announce [--announce_all] [ blob_hash | --blob_hash= blob_hash ]\n [ stream_hash | --stream_hash= stream_hash ]\n [ sd_hash | --sd_hash= sd_hash ]\n\n\nOptions:\n --announce_all= announce_all : (bool) announce all the blobs possessed by user\n --blob_hash= blob_hash : (str) announce a blob, specified by blob_hash\n --stream_hash= stream_hash : (str) announce all blobs associated with\n stream_hash\n --sd_hash= sd_hash : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful",
"text": "Announce blobs to the DHT\n\nUsage:\n blob_announce [ blob_hash | --blob_hash= blob_hash ]\n [ stream_hash | --stream_hash= stream_hash ] | [ sd_hash | --sd_hash= sd_hash ]\n\n\nOptions:\n --blob_hash= blob_hash : (str) announce a blob, specified by blob_hash\n --stream_hash= stream_hash : (str) announce all blobs associated with\n stream_hash\n --sd_hash= sd_hash : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful",
"title": "blob_announce"
},
{
@@ -310,6 +325,11 @@
"text": "Returns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nUsage:\n blob_list [--needed] [--finished] [ uri | --uri= uri ]\n [ stream_hash | --stream_hash= stream_hash ]\n [ sd_hash | --sd_hash= sd_hash ]\n [ page_size | --page_size= page_size ]\n [ page | --page= page ]\n\n\nOptions:\n --needed : (bool) only return needed blobs\n --finished : (bool) only return finished blobs\n --uri= uri : (str) filter blobs by stream in a uri\n --stream_hash= stream_hash : (str) filter blobs by stream hash\n --sd_hash= sd_hash : (str) filter blobs by sd hash\n --page_size= page_size : (int) results page size\n --page= page : (int) page of results to return\n\nReturns:\n (list) List of blob hashes",
"title": "blob_list"
},
{
"location": "/cli/#blob_reflect",
"text": "Reflects specified blobs\n\nUsage:\n blob_reflect ( blob_hashes ...) [--reflector_server= reflector_server ]\n\n\nOptions:\n --reflector_server= reflector_server : (str) reflector address\n\nReturns:\n (list) reflected blob hashes",
"title": "blob_reflect"
},
{
"location": "/cli/#blob_reflect_all",
"text": "Reflects all saved blobs\n\nUsage:\n blob_reflect_all\n\n\nOptions:\n None\n\nReturns:\n (bool) true if successful",
@@ -327,7 +347,7 @@
},
{
"location": "/cli/#channel_import",
"text": "Import serialized channel signing information (to allow signing new claims to the channel)\n\nUsage:\n channel_import ( serialized_certificate_info |\n --serialized_certificate_info= serialized_certificate_info )\n\n\nOptions:\n --serialized_certificate_info= serialized_certificate_info : (str) certificate info\n\nReturns:\n (dict) Result dictionary",
"text": "Import serialized channel signing information (to allow signing new claims to the channel)\n\nUsage:\n channel_import ( serialized_certificate_info | --serialized_certificate_info= serialized_certificate_info )\n\n\nOptions:\n --serialized_certificate_info= serialized_certificate_info : (str) certificate info\n\nReturns:\n (dict) Result dictionary",
"title": "channel_import"
},
{
@@ -357,7 +377,7 @@
},
{
"location": "/cli/#claim_list_mine",
"text": "List my name claims\n\nUsage:\n claim_list_mine\n\n\nOptions:\n None\n\nReturns:\n (list) List of name claims owned by user\n [\n {\n 'address': (str) address that owns the claim\n 'amount': (float) amount assigned to the claim\n 'blocks_to_expiration': (int) number of blocks until it expires\n 'category': (str) claim , update , or support \n 'claim_id': (str) claim ID of the claim\n 'confirmations': (int) number of blocks of confirmations for the claim\n 'expiration_height': (int) the block height which the claim will expire\n 'expired': (bool) true if expired, false otherwise\n 'height': (int) height of the block containing the claim\n 'is_spent': (bool) true if claim is abandoned, false otherwise\n 'name': (str) name of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'txid': (str) txid of the cliam\n 'nout': (int) nout of the claim\n 'value': (str) value of the claim\n },\n ]",
"text": "List my name claims\n\nUsage:\n claim_list_mine\n\n\nOptions:\n None\n\nReturns:\n (list) List of name claims owned by user\n [\n {\n 'address': (str) address that owns the claim\n 'amount': (float) amount assigned to the claim\n 'blocks_to_expiration': (int) number of blocks until it expires\n 'category': (str) claim , update , or support \n 'claim_id': (str) claim ID of the claim\n 'confirmations': (int) number of blocks of confirmations for the claim\n 'expiration_height': (int) the block height which the claim will expire\n 'expired': (bool) true if expired, false otherwise\n 'height': (int) height of the block containing the claim\n 'is_spent': (bool) true if claim is abandoned, false otherwise\n 'name': (str) name of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'txid': (str) txid of the claim\n 'nout': (int) nout of the claim\n 'value': (str) value of the claim\n },\n ]",
"title": "claim_list_mine"
},
{
@@ -382,7 +402,7 @@
},
{
"location": "/cli/#cli_test_command",
"text": "This command is only for testing the CLI argument parsing\nUsage:\n cli_test_command [--a_arg] [--b_arg] ( pos_arg | --pos_arg= pos_arg )\n [ pos_args ...] [--pos_arg2= pos_arg2 ]\n [--pos_arg3= pos_arg3 ]\n\n\nOptions:\n --a_arg : a arg\n --b_arg : b arg\n --pos_arg= pos_arg : pos arg\n --pos_args= pos_args : pos args\n --pos_arg2= pos_arg2 : pos arg 2\n --pos_arg3= pos_arg3 : pos arg 3\n\nReturns:\n pos args",
"text": "This command is only for testing the CLI argument parsing\nUsage:\n cli_test_command [--a_arg] [--b_arg] ( pos_arg | --pos_arg= pos_arg )\n [ pos_args ...] [--pos_arg2= pos_arg2 ]\n [--pos_arg3= pos_arg3 ]\n\n\nOptions:\n --a_arg : (bool) a arg\n --b_arg : (bool) b arg\n --pos_arg= pos_arg : (int) pos arg\n --pos_args= pos_args : (int) pos args\n --pos_arg2= pos_arg2 : (int) pos arg 2\n --pos_arg3= pos_arg3 : (int) pos arg 3\n\nReturns:\n pos args",
"title": "cli_test_command"
},
{
@@ -402,7 +422,7 @@
},
{
"location": "/cli/#file_list",
"text": "List files limited by optional filters\n\nUsage:\n file_list [--sd_hash= sd_hash ] [--file_name= file_name ] [--stream_hash= stream_hash ]\n [--rowid= rowid ] [--claim_id= claim_id ] [--outpoint= outpoint ] [--txid= txid ] [--nout= nout ]\n [--channel_claim_id= channel_claim_id ] [--channel_name= channel_name ]\n [--claim_name= claim_name ] [--full_status]\n\n\nOptions:\n --sd_hash= sd_hash : (str) get file with matching sd hash\n --file_name= file_name : (str) get file with matching file name in the\n downloads folder\n --stream_hash= stream_hash : (str) get file with matching stream hash\n --rowid= rowid : (int) get file with matching row id\n --claim_id= claim_id : (str) get file with matching claim id\n --outpoint= outpoint : (str) get file with matching claim outpoint\n --txid= txid : (str) get file with matching claim txid\n --nout= nout : (int) get file with matching claim nout\n --channel_claim_id= channel_claim_id : (str) get file with matching channel claim id\n --channel_name= channel_name : (str) get file with matching channel name\n --claim_name= claim_name : (str) get file with matching claim name\n --full_status : (bool) full status, populate the\n 'message' and 'size' fields\n\nReturns:\n (list) List of files\n\n [\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) None if full_status is false or if claim is not found,\n 'outpoint': (str) None if full_status is false or if claim is not found,\n 'txid': (str) None if full_status is false or if claim is not found,\n 'nout': (int) None if full_status is false or if claim is not found,\n 'metadata': (dict) None if full_status is false or if claim is not found,\n 'channel_claim_id': (str) None if full_status is false or if claim is not found or signed,\n 'channel_name': (str) None if full_status is false or if claim is not found or signed,\n 'claim_name': (str) None if full_status is false or if claim is not found\n },\n ]",
"text": "List files limited by optional filters\n\nUsage:\n file_list [--sd_hash= sd_hash ] [--file_name= file_name ] [--stream_hash= stream_hash ]\n [--rowid= rowid ] [--claim_id= claim_id ] [--outpoint= outpoint ] [--txid= txid ] [--nout= nout ]\n [--channel_claim_id= channel_claim_id ] [--channel_name= channel_name ]\n [--claim_name= claim_name ] [--full_status] [--sort= sort_method ...]\n\n\nOptions:\n --sd_hash= sd_hash : (str) get file with matching sd hash\n --file_name= file_name : (str) get file with matching file name in the\n downloads folder\n --stream_hash= stream_hash : (str) get file with matching stream hash\n --rowid= rowid : (int) get file with matching row id\n --claim_id= claim_id : (str) get file with matching claim id\n --outpoint= outpoint : (str) get file with matching claim outpoint\n --txid= txid : (str) get file with matching claim txid\n --nout= nout : (int) get file with matching claim nout\n --channel_claim_id= channel_claim_id : (str) get file with matching channel claim id\n --channel_name= channel_name : (str) get file with matching channel name\n --claim_name= claim_name : (str) get file with matching claim name\n --full_status : (bool) full status, populate the\n 'message' and 'size' fields\n --sort= sort_method : (str) sort by any property, like 'file_name'\n or 'metadata.author'; to specify direction\n append ',asc' or ',desc'\n\nReturns:\n (list) List of files\n\n [\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) None if full_status is false or if claim is not found,\n 'outpoint': (str) None if full_status is false or if claim is not found,\n 'txid': (str) None if full_status is false or if claim is not found,\n 'nout': (int) None if full_status is false or if claim is not found,\n 'metadata': (dict) None if full_status is false or if claim is not found,\n 'channel_claim_id': (str) None if full_status is false or if claim is not found or signed,\n 'channel_name': (str) None if full_status is false or if claim is not found or signed,\n 'claim_name': (str) None if full_status is false or if claim is not found\n },\n ]",
"title": "file_list"
},
{
@@ -427,14 +447,24 @@
},
{
"location": "/cli/#peer_list",
"text": "Get peers for blob hash\n\nUsage:\n peer_list ( blob_hash | --blob_hash= blob_hash ) [ timeout | --timeout= timeout ]\n\n\nOptions:\n --blob_hash= blob_hash : (str) find available peers for this blob hash\n --timeout= timeout : (int) peer search timeout in seconds\n\nReturns:\n (list) List of contacts",
"text": "Get peers for blob hash\n\nUsage:\n peer_list ( blob_hash | --blob_hash= blob_hash ) [ timeout | --timeout= timeout ]\n\n\nOptions:\n --blob_hash= blob_hash : (str) find available peers for this blob hash\n --timeout= timeout : (int) peer search timeout in seconds\n\nReturns:\n (list) List of contact dictionaries {'host': peer ip , 'port': peer port , 'node_id': peer node id }",
"title": "peer_list"
},
{
"location": "/cli/#peer_ping",
"text": "Send a kademlia ping to the specified peer. If address and port are provided the peer is directly pinged,\nif not provided the peer is located first.\n\nUsage:\n peer_ping ( node_id | --node_id= node_id ) [ address | --address= address ] [ port | --port= port ]\n\n\nOptions:\n --address= address : (str) ip address of the peer\n --port= port : (int) udp port of the peer\n\nReturns:\n (str) pong, or {'error': error message } if an error is encountered",
"title": "peer_ping"
},
{
"location": "/cli/#publish",
"text": "Make a new name claim and publish associated data to lbrynet,\nupdate over existing claim if user already has a claim for name.\n\nFields required in the final Metadata are:\n 'title'\n 'description'\n 'author'\n 'language'\n 'license'\n 'nsfw'\n\nMetadata can be set by either using the metadata argument or by setting individual arguments\nfee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\nor sources. Individual arguments will overwrite the fields specified in metadata argument.\n\nUsage:\n publish ( name | --name= name ) ( bid | --bid= bid ) [--metadata= metadata ]\n [--file_path= file_path ] [--fee= fee ] [--title= title ]\n [--description= description ] [--author= author ] [--language= language ]\n [--license= license ] [--license_url= license_url ] [--thumbnail= thumbnail ]\n [--preview= preview ] [--nsfw= nsfw ] [--sources= sources ]\n [--channel_name= channel_name ] [--channel_id= channel_id ]\n [--claim_address= claim_address ] [--change_address= change_address ]\n\n\nOptions:\n --name= name : (str) name of the content\n --bid= bid : (float) amount to back the claim\n --metadata= metadata : (dict) ClaimDict to associate with the claim.\n --file_path= file_path : (str) path to file to be associated with name. If provided,\n a lbry stream of this file will be used in 'sources'.\n If no path is given but a sources dict is provided,\n it will be used. If neither are provided, an\n error is raised.\n --fee= fee : (dict) Dictionary representing key fee to download content:\n {\n 'currency': currency_symbol,\n 'amount': float,\n 'address': str, optional\n }\n supported currencies: LBC, USD, BTC\n If an address is not provided a new one will be\n automatically generated. Default fee is zero.\n --title= title : (str) title of the publication\n --description= description : (str) description of the publication\n --author= author : (str) author of the publication\n --language= language : (str) language of the publication\n --license= license : (str) publication license\n --license_url= license_url : (str) publication license url\n --thumbnail= thumbnail : (str) thumbnail url\n --preview= preview : (str) preview url\n --nsfw= nsfw : (bool) title of the publication\n --sources= sources : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file\n --channel_name= channel_name : (str) name of the publisher channel name in the wallet\n --channel_id= channel_id : (str) claim id of the publisher channel, does not check\n for channel claim being in the wallet. This allows\n publishing to a channel where only the certificate\n private key is in the wallet.\n --claim_address= claim_address : (str) address where the claim is sent to, if not specified\n new address wil automatically be created\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"text": "Make a new name claim and publish associated data to lbrynet,\nupdate over existing claim if user already has a claim for name.\n\nFields required in the final Metadata are:\n 'title'\n 'description'\n 'author'\n 'language'\n 'license'\n 'nsfw'\n\nMetadata can be set by either using the metadata argument or by setting individual arguments\nfee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\nor sources. Individual arguments will overwrite the fields specified in metadata argument.\n\nUsage:\n publish ( name | --name= name ) ( bid | --bid= bid ) [--metadata= metadata ]\n [--file_path= file_path ] [--fee= fee ] [--title= title ]\n [--description= description ] [--author= author ] [--language= language ]\n [--license= license ] [--license_url= license_url ] [--thumbnail= thumbnail ]\n [--preview= preview ] [--nsfw= nsfw ] [--sources= sources ]\n [--channel_name= channel_name ] [--channel_id= channel_id ]\n [--claim_address= claim_address ] [--change_address= change_address ]\n\n\nOptions:\n --name= name : (str) name of the content\n --bid= bid : (decimal) amount to back the claim\n --metadata= metadata : (dict) ClaimDict to associate with the claim.\n --file_path= file_path : (str) path to file to be associated with name. If provided,\n a lbry stream of this file will be used in 'sources'.\n If no path is given but a sources dict is provided,\n it will be used. If neither are provided, an\n error is raised.\n --fee= fee : (dict) Dictionary representing key fee to download content:\n {\n 'currency': currency_symbol,\n 'amount': decimal,\n 'address': str, optional\n }\n supported currencies: LBC, USD, BTC\n If an address is not provided a new one will be\n automatically generated. Default fee is zero.\n --title= title : (str) title of the publication\n --description= description : (str) description of the publication\n --author= author : (str) author of the publication\n --language= language : (str) language of the publication\n --license= license : (str) publication license\n --license_url= license_url : (str) publication license url\n --thumbnail= thumbnail : (str) thumbnail url\n --preview= preview : (str) preview url\n --nsfw= nsfw : (bool) whether the content is nsfw\n --sources= sources : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file\n --channel_name= channel_name : (str) name of the publisher channel name in the wallet\n --channel_id= channel_id : (str) claim id of the publisher channel, does not check\n for channel claim being in the wallet. This allows\n publishing to a channel where only the certificate\n private key is in the wallet.\n --claim_address= claim_address : (str) address where the claim is sent to, if not specified\n new address will automatically be created\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (decimal) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "publish"
},
{
"location": "/cli/#report_bug",
"text": "Report a bug to slack\n\nUsage:\n report_bug ( message | --message= message )\n\n\nOptions:\n --message= message : (str) Description of the bug\n\nReturns:\n (bool) true if successful",
"title": "report_bug"
},
{
"location": "/cli/#resolve",
"text": "Resolve given LBRY URIs\n\nUsage:\n resolve [--force] ( uri | --uri= uri ) [ uris ...]\n\n\nOptions:\n --force : (bool) force refresh and ignore cache\n --uri= uri : (str) uri to resolve\n --uris= uris : (list) uris to resolve\n\nReturns:\n Dictionary of results, keyed by uri\n ' uri ': {\n If a resolution error occurs:\n 'error': Error message\n\n If the uri resolves to a channel or a claim in a channel:\n 'certificate': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the certificate claim,\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}],\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n\n If the uri resolves to a channel:\n 'claims_in_channel': (int) number of claims in the channel,\n\n If the uri resolves to a claim:\n 'claim': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the claim,\n 'channel_name': (str) channel name if claim is in a channel\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}]\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n }",
@@ -447,7 +477,7 @@
},
{
"location": "/cli/#routing_table_get",
"text": "Get DHT routing information\n\nUsage:\n routing_table_get\n\n\nOptions:\n None\n\nReturns:\n (dict) dictionary containing routing and contact information\n {\n buckets : {\n bucket index : [\n {\n address : (str) peer address,\n node_id : (str) peer node id,\n blobs : (list) blob hashes announced by peer\n }\n ]\n },\n contacts : (list) contact node ids,\n blob_hashes : (list) all of the blob hashes stored by peers in the list of buckets,\n node_id : (str) the local dht node id\n }",
"text": "Get DHT routing information\n\nUsage:\n routing_table_get\n\n\nOptions:\n None\n\nReturns:\n (dict) dictionary containing routing and contact information\n {\n buckets : {\n bucket index : [\n {\n address : (str) peer address,\n port : (int) peer udp port\n node_id : (str) peer node id,\n blobs : (list) blob hashes announced by peer\n }\n ]\n },\n contacts : (list) contact node ids,\n blob_hashes : (list) all of the blob hashes stored by peers in the list of buckets,\n node_id : (str) the local dht node id\n }",
"title": "routing_table_get"
},
{
@@ -462,12 +492,12 @@
},
{
"location": "/cli/#status",
"text": "Get daemon status\n\nUsage:\n status [--session_status] [--dht_status]\n\n\nOptions:\n --session_status : (bool) include session status in results\n --dht_status : (bool) include dht network and peer status\n\nReturns:\n (dict) lbrynet-daemon status\n {\n 'lbry_id': lbry peer id, base58,\n 'installation_id': installation id, base58,\n 'is_running': bool,\n 'is_first_run': bool,\n 'startup_status': {\n 'code': status code,\n 'message': status message\n },\n 'connection_status': {\n 'code': connection status code,\n 'message': connection status message\n },\n 'blockchain_status': {\n 'blocks': local blockchain height,\n 'blocks_behind': remote_height - local_height,\n 'best_blockhash': block hash of most recent block,\n },\n 'wallet_is_encrypted': bool,\n\n If given the session status option:\n 'session_status': {\n 'managed_blobs': count of blobs in the blob manager,\n 'managed_streams': count of streams in the file manager\n 'announce_queue_size': number of blobs currently queued to be announced\n 'should_announce_blobs': number of blobs that should be announced\n }\n\n If given the dht status option:\n 'dht_status': {\n 'kbps_received': current kbps receiving,\n 'kbps_sent': current kdps being sent,\n 'total_bytes_sent': total bytes sent,\n 'total_bytes_received': total bytes received,\n 'queries_received': number of queries received per second,\n 'queries_sent': number of queries sent per second,\n 'recent_contacts': count of recently contacted peers,\n 'unique_contacts': count of unique peers\n },\n }",
"text": "Get daemon status\n\nUsage:\n status\n\n\nOptions:\n None\n\nReturns:\n (dict) lbrynet-daemon status\n {\n 'installation_id': (str) installation id - base58,\n 'is_running': (bool),\n 'is_first_run': bool,\n 'skipped_components': (list) [names of skipped components (str)],\n 'startup_status': { Does not include components which have been skipped\n 'database': (bool),\n 'wallet': (bool),\n 'session': (bool),\n 'dht': (bool),\n 'hash_announcer': (bool),\n 'stream_identifier': (bool),\n 'file_manager': (bool),\n 'blob_manager': (bool),\n 'blockchain_headers': (bool),\n 'peer_protocol_server': (bool),\n 'reflector': (bool),\n 'upnp': (bool),\n 'exchange_rate_manager': (bool),\n },\n 'connection_status': {\n 'code': (str) connection status code,\n 'message': (str) connection status message\n },\n 'blockchain_headers': {\n 'downloading_headers': (bool),\n 'download_progress': (float) 0-100.0\n },\n 'wallet': {\n 'blocks': (int) local blockchain height,\n 'blocks_behind': (int) remote_height - local_height,\n 'best_blockhash': (str) block hash of most recent block,\n 'is_encrypted': (bool),\n 'is_locked': (bool),\n },\n 'dht': {\n 'node_id': (str) lbry dht node id - hex encoded,\n 'peers_in_routing_table': (int) the number of peers in the routing table,\n },\n 'blob_manager': {\n 'finished_blobs': (int) number of finished blobs in the blob manager,\n },\n 'hash_announcer': {\n 'announce_queue_size': (int) number of blobs currently queued to be announced\n },\n 'file_manager': {\n 'managed_files': (int) count of files in the file manager,\n }\n }",
"title": "status"
},
{
"location": "/cli/#stream_availability",
"text": "Get stream availability for lbry uri\n\nUsage:\n stream_availability ( uri | --uri= uri )\n [ search_timeout | --search_timeout= search_timeout ]\n [ blob_timeout | --blob_timeout= blob_timeout ]\n\n\nOptions:\n --uri= uri : (str) check availability for this uri\n --search_timeout= search_timeout : (int) how long to search for peers for the blob\n in the dht\n --search_timeout= blob_timeout : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n 'is_available': bool ,\n 'did_decode': bool ,\n 'did_resolve': bool ,\n 'is_stream': bool ,\n 'num_blobs_in_stream': int ,\n 'sd_hash': str ,\n 'sd_blob_availability': dict see `blob_availability`,\n 'head_blob_hash': str ,\n 'head_blob_availability': dict see `blob_availability`,\n 'use_upnp': bool ,\n 'upnp_redirect_is_set': bool ,\n 'error': None | str error message\n }",
"text": "Get stream availability for lbry uri\n\nUsage:\n stream_availability ( uri | --uri= uri )\n [ search_timeout | --search_timeout= search_timeout ]\n [ blob_timeout | --blob_timeout= blob_timeout ]\n\n\nOptions:\n --uri= uri : (str) check availability for this uri\n --search_timeout= search_timeout : (int) how long to search for peers for the blob\n in the dht\n --blob_timeout= blob_timeout : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n 'is_available': bool ,\n 'did_decode': bool ,\n 'did_resolve': bool ,\n 'is_stream': bool ,\n 'num_blobs_in_stream': int ,\n 'sd_hash': str ,\n 'sd_blob_availability': dict see `blob_availability`,\n 'head_blob_hash': str ,\n 'head_blob_availability': dict see `blob_availability`,\n 'use_upnp': bool ,\n 'upnp_redirect_is_set': bool ,\n 'error': None | str error message\n }",
"title": "stream_availability"
},
{
@@ -537,7 +567,7 @@
},
{
"location": "/cli/#wallet_send",
"text": "Send credits. If given an address, send credits to it. If given a claim id, send a tip\nto the owner of a claim specified by uri. A tip is a claim support where the recipient\nof the support is the claim address for the claim being supported.\n\nUsage:\n wallet_send ( amount | --amount= amount )\n (( address | --address= address ) | ( claim_id | --claim_id= claim_id ))\n\n\nOptions:\n --amount= amount : (float) amount of credit to send\n --address= address : (str) address to send credits to\n --claim_id= claim_id : (float) claim_id of the claim to send to tip to\n\nReturns:\n If sending to an address:\n (bool) true if payment successfully scheduled\n\n If sending a claim tip:\n (dict) Dictionary containing the result of the support\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"text": "Send credits. If given an address, send credits to it. If given a claim id, send a tip\nto the owner of a claim specified by uri. A tip is a claim support where the recipient\nof the support is the claim address for the claim being supported.\n\nUsage:\n wallet_send ( amount | --amount= amount )\n (( address | --address= address ) | ( claim_id | --claim_id= claim_id ))\n\n\nOptions:\n --amount= amount : (Decimal) amount of credit to send\n --address= address : (str) address to send credits to\n --claim_id= claim_id : (str) claim_id of the claim to send to tip to\n\nReturns:\n If sending to an address:\n (bool) true if payment successfully scheduled\n\n If sending a claim tip:\n (dict) Dictionary containing the result of the support\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"title": "wallet_send"
},
{
+23
View File
@@ -0,0 +1,23 @@
# This is an example daemon_settings.yml file.
# See https://lbry.tech/resources/daemon-settings for all configuration keys and values
share_usage_data: True
lbryum_servers:
- lbryumx1.lbry.io:50001
- lbryumx2.lbry.io:50001
blockchain_name: lbrycrd_main
data_dir: /home/lbry/.lbrynet
download_directory: /home/lbry/downloads
delete_blobs_on_remove: True
dht_node_port: 4444
peer_port: 3333
use_upnp: True
#components_to_skip:
# - peer_protocol_server
# - hash_announcer
# - dht
File renamed without changes.

Before

Width:  |  Height:  |  Size: 7.4 KiB

After

Width:  |  Height:  |  Size: 7.4 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 19 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 1.2 KiB

After

Width:  |  Height:  |  Size: 1.2 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 2.6 KiB

After

Width:  |  Height:  |  Size: 2.6 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 6.1 KiB

After

Width:  |  Height:  |  Size: 6.1 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 97 KiB

After

Width:  |  Height:  |  Size: 97 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 1.1 KiB

After

Width:  |  Height:  |  Size: 1.1 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 361 KiB

After

Width:  |  Height:  |  Size: 361 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 5.3 KiB

After

Width:  |  Height:  |  Size: 5.3 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 15 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 31 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 KiB

+2 -4
View File
@@ -1,6 +1,4 @@
import logging
__version__ = "0.19.1rc2"
__name__ = "lbrynet"
__version__ = "0.31.0rc3"
version = tuple(__version__.split('.'))
logging.getLogger(__name__).addHandler(logging.NullHandler())
-268
View File
@@ -1,268 +0,0 @@
import collections
import logging
from twisted.internet import defer, task
from requests import auth
from txrequests import Session
from lbrynet import conf
from lbrynet.core import looping_call_manager, utils, system_info
# Things We Track
SERVER_STARTUP = 'Server Startup'
SERVER_STARTUP_SUCCESS = 'Server Startup Success'
SERVER_STARTUP_ERROR = 'Server Startup Error'
DOWNLOAD_STARTED = 'Download Started'
DOWNLOAD_ERRORED = 'Download Errored'
DOWNLOAD_FINISHED = 'Download Finished'
HEARTBEAT = 'Heartbeat'
CLAIM_ACTION = 'Claim Action' # publish/create/update/abandon
NEW_CHANNEL = 'New Channel'
CREDITS_SENT = 'Credits Sent'
BLOB_BYTES_UPLOADED = 'Blob Bytes Uploaded'
log = logging.getLogger(__name__)
class Manager(object):
def __init__(self, analytics_api, context=None, installation_id=None, session_id=None):
self.analytics_api = analytics_api
self._tracked_data = collections.defaultdict(list)
self.looping_call_manager = self._setup_looping_calls()
self.context = context or self._make_context(
system_info.get_platform(), conf.settings['wallet'])
self.installation_id = installation_id or conf.settings.installation_id
self.session_id = session_id or conf.settings.get_session_id()
self.is_started = False
@classmethod
def new_instance(cls, enabled=None):
api = Api.new_instance(enabled)
return cls(api)
# Things We Track
def send_server_startup(self):
self.analytics_api.track(self._event(SERVER_STARTUP))
def send_server_startup_success(self):
self.analytics_api.track(self._event(SERVER_STARTUP_SUCCESS))
def send_server_startup_error(self, message):
self.analytics_api.track(self._event(SERVER_STARTUP_ERROR, {'message': message}))
def send_download_started(self, id_, name, claim_dict=None):
self.analytics_api.track(
self._event(DOWNLOAD_STARTED, self._download_properties(id_, name, claim_dict))
)
def send_download_errored(self, err, id_, name, claim_dict, report):
download_error_properties = self._download_error_properties(err, id_, name, claim_dict,
report)
self.analytics_api.track(self._event(DOWNLOAD_ERRORED, download_error_properties))
def send_download_finished(self, id_, name, report, claim_dict=None):
download_properties = self._download_properties(id_, name, claim_dict, report)
self.analytics_api.track(self._event(DOWNLOAD_FINISHED, download_properties))
def send_claim_action(self, action):
self.analytics_api.track(self._event(CLAIM_ACTION, {'action': action}))
def send_new_channel(self):
self.analytics_api.track(self._event(NEW_CHANNEL))
def send_credits_sent(self):
self.analytics_api.track(self._event(CREDITS_SENT))
def _send_heartbeat(self):
self.analytics_api.track(self._event(HEARTBEAT))
def _update_tracked_metrics(self):
should_send, value = self.summarize_and_reset(BLOB_BYTES_UPLOADED)
if should_send:
self.analytics_api.track(self._metric_event(BLOB_BYTES_UPLOADED, value))
# Setup / Shutdown
def start(self):
if not self.is_started:
for name, _, interval in self._get_looping_calls():
self.looping_call_manager.start(name, interval)
self.is_started = True
def shutdown(self):
self.looping_call_manager.shutdown()
def register_repeating_metric(self, event_name, value_generator, frequency=300):
lcall = task.LoopingCall(self._send_repeating_metric, event_name, value_generator)
self.looping_call_manager.register_looping_call(event_name, lcall)
lcall.start(frequency)
def _get_looping_calls(self):
return [
('send_heartbeat', self._send_heartbeat, 60),
('update_tracked_metrics', self._update_tracked_metrics, 300),
]
def _setup_looping_calls(self):
call_manager = looping_call_manager.LoopingCallManager()
for name, fn, _ in self._get_looping_calls():
call_manager.register_looping_call(name, task.LoopingCall(fn))
return call_manager
def _send_repeating_metric(self, event_name, value_generator):
result = value_generator()
self._if_deferred(result, self._send_repeating_metric_value, event_name)
def _send_repeating_metric_value(self, result, event_name):
should_send, value = result
if should_send:
self.analytics_api.track(self._metric_event(event_name, value))
def add_observation(self, metric, value):
self._tracked_data[metric].append(value)
def summarize_and_reset(self, metric, op=sum):
"""Apply `op` on the current values for `metric`.
This operation also resets the metric.
Returns:
a tuple (should_send, value)
"""
try:
values = self._tracked_data.pop(metric)
return True, op(values)
except KeyError:
return False, None
def _event(self, event, event_properties=None):
return {
'userId': 'lbry',
'event': event,
'properties': self._event_properties(event_properties),
'context': self.context,
'timestamp': utils.isonow()
}
def _metric_event(self, metric_name, value):
return self._event(metric_name, {'value': value})
def _event_properties(self, event_properties=None):
properties = {
'lbry_id': self.installation_id,
'session_id': self.session_id,
}
properties.update(event_properties or {})
return properties
@staticmethod
def _download_properties(id_, name, claim_dict=None, report=None):
sd_hash = None if not claim_dict else claim_dict.source_hash
p = {
'download_id': id_,
'name': name,
'stream_info': sd_hash
}
if report:
p['report'] = report
return p
@staticmethod
def _download_error_properties(error, id_, name, claim_dict, report):
def error_name(err):
if not hasattr(type(err), "__name__"):
return str(type(err))
return type(err).__name__
return {
'download_id': id_,
'name': name,
'stream_info': claim_dict.source_hash,
'error': error_name(error),
'reason': error.message,
'report': report
}
@staticmethod
def _make_context(platform, wallet):
return {
'app': {
'name': 'lbrynet',
'version': platform['lbrynet_version'],
'python_version': platform['python_version'],
'build': platform['build'],
'wallet': {
'name': wallet,
'version': platform['lbryum_version'] if wallet == conf.LBRYUM_WALLET else None
},
},
# TODO: expand os info to give linux/osx specific info
'os': {
'name': platform['os_system'],
'version': platform['os_release']
},
'library': {
'name': 'lbrynet-analytics',
'version': '1.0.0'
},
}
@staticmethod
def _if_deferred(maybe_deferred, callback, *args, **kwargs):
if isinstance(maybe_deferred, defer.Deferred):
maybe_deferred.addCallback(callback, *args, **kwargs)
else:
callback(maybe_deferred, *args, **kwargs)
class Api(object):
def __init__(self, session, url, write_key, enabled):
self.session = session
self.url = url
self._write_key = write_key
self._enabled = enabled
def _post(self, endpoint, data):
# there is an issue with a timing condition with keep-alive
# that is best explained here: https://github.com/mikem23/keepalive-race
#
# If you make a request, wait just the right amount of time,
# then make another request, the requests module may opt to
# reuse the connection, but by the time the server gets it the
# timeout will have expired.
#
# by forcing the connection to close, we will disable the keep-alive.
assert endpoint[0] == '/'
headers = {"Connection": "close"}
return self.session.post(
self.url + endpoint,
json=data,
auth=auth.HTTPBasicAuth(self._write_key, ''),
headers=headers
)
def track(self, event):
"""Send a single tracking event"""
if not self._enabled:
return defer.succeed('Analytics disabled')
def _log_error(failure, event):
log.warning('Failed to send track event. %s (%s)', failure.getTraceback(), str(event))
log.debug('Sending track event: %s', event)
d = self._post('/track', event)
d.addErrback(_log_error, event)
return d
@classmethod
def new_instance(cls, enabled=None):
"""Initialize an instance using values from the configuration"""
session = Session()
if enabled is None:
enabled = conf.settings['share_usage_data']
return cls(
session,
conf.settings['ANALYTICS_ENDPOINT'],
utils.deobfuscate(conf.settings['ANALYTICS_TOKEN']),
enabled,
)
-1
View File
@@ -1 +0,0 @@
import paths
-26
View File
@@ -1,26 +0,0 @@
# App root files dir
# https://developer.android.com/reference/android/content/ContextWrapper.html#getFilesDir%28%29
def android_files_dir():
return None
# Base internal storage path
def android_internal_storage_dir():
return None
# Base external (SD card, if present) storage path
def android_external_storage_dir():
return None
# Internal device storage (private app folder)
# https://developer.android.com/reference/android/content/ContextWrapper.html#getExternalFilesDirs(java.lang.String)
def android_app_internal_storage_dir():
return None
# External (app folder on SD card, if present) storage
# https://developer.android.com/reference/android/content/ContextWrapper.html#getExternalFilesDirs(java.lang.String)
def android_app_external_storage_dir():
return None
+6 -4
View File
@@ -1,4 +1,6 @@
from blob_file import BlobFile
from creator import BlobFileCreator
from writer import HashBlobWriter
from reader import HashBlobReader
from lbrynet.cryptoutils import get_lbry_hash_obj
MAX_BLOB_SIZE = 2 * 2 ** 20
# digest_size is in bytes, and blob hashes are hex encoded
blobhash_length = get_lbry_hash_obj().digest_size * 2
+145 -166
View File
@@ -1,19 +1,48 @@
import logging
import os
from twisted.internet import defer, threads
from twisted.web.client import FileBodyProducer
from twisted.python.failure import Failure
from lbrynet.core.Error import DownloadCanceledError, InvalidDataError, InvalidBlobHashError
from lbrynet.core.utils import is_valid_blobhash
import asyncio
import binascii
import logging
import typing
from cryptography.hazmat.primitives.ciphers import Cipher, modes
from cryptography.hazmat.primitives.ciphers.algorithms import AES
from cryptography.hazmat.primitives.padding import PKCS7
from lbrynet.cryptoutils import backend, get_lbry_hash_obj
from lbrynet.error import DownloadCancelledError, InvalidBlobHashError, InvalidDataError
from lbrynet.blob import MAX_BLOB_SIZE, blobhash_length
from lbrynet.blob.blob_info import BlobInfo
from lbrynet.blob.writer import HashBlobWriter
from lbrynet.blob.reader import HashBlobReader
log = logging.getLogger(__name__)
MAX_BLOB_SIZE = 2 * 2 ** 20
def is_valid_hashcharacter(char: str) -> bool:
return char in "0123456789abcdef"
class BlobFile(object):
def is_valid_blobhash(blobhash: str) -> bool:
"""Checks whether the blobhash is the correct length and contains only
valid characters (0-9, a-f)
@param blobhash: string, the blobhash to check
@return: True/False
"""
return len(blobhash) == blobhash_length and all(is_valid_hashcharacter(l) for l in blobhash)
def encrypt_blob_bytes(key: bytes, iv: bytes, unencrypted: bytes) -> typing.Tuple[bytes, str]:
cipher = Cipher(AES(key), modes.CBC(iv), backend=backend)
padder = PKCS7(AES.block_size).padder()
encryptor = cipher.encryptor()
encrypted = encryptor.update(padder.update(unencrypted) + padder.finalize()) + encryptor.finalize()
digest = get_lbry_hash_obj()
digest.update(encrypted)
return encrypted, digest.hexdigest()
class BlobFile:
"""
A chunk of data available on the network which is specified by a hashsum
@@ -22,187 +51,137 @@ class BlobFile(object):
Also can be used for reading from blobs on the local filesystem
"""
def __str__(self):
return self.blob_hash[:16]
def __repr__(self):
return '<{}({})>'.format(self.__class__.__name__, str(self))
def __init__(self, blob_dir, blob_hash, length=None):
def __init__(self, loop: asyncio.BaseEventLoop, blob_dir: str, blob_hash: str,
length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['BlobFile'], typing.Awaitable]] = None):
if not is_valid_blobhash(blob_hash):
raise InvalidBlobHashError(blob_hash)
self.loop = loop
self.blob_hash = blob_hash
self.length = length
self.writers = {} # {Peer: writer, finished_deferred}
self._verified = False
self.readers = 0
self.blob_dir = blob_dir
self.file_path = os.path.join(blob_dir, self.blob_hash)
self.blob_write_lock = defer.DeferredLock()
self.writers: typing.List[HashBlobWriter] = []
self.verified: asyncio.Event = asyncio.Event(loop=self.loop)
self.finished_writing = asyncio.Event(loop=loop)
self.blob_write_lock = asyncio.Lock(loop=loop)
if os.path.isfile(os.path.join(blob_dir, blob_hash)):
length = length or int(os.stat(os.path.join(blob_dir, blob_hash)).st_size)
self.length = length
self.verified.set()
self.finished_writing.set()
self.saved_verified_blob = False
if os.path.isfile(self.file_path):
self.set_length(os.path.getsize(self.file_path))
# This assumes that the hash of the blob has already been
# checked as part of the blob creation process. It might
# be worth having a function that checks the actual hash;
# its probably too expensive to have that check be part of
# this call.
self._verified = True
self.blob_completed_callback = blob_completed_callback
def open_for_writing(self, peer):
def writer_finished(self, writer: HashBlobWriter):
def callback(finished: asyncio.Future):
try:
error = finished.result()
except Exception as err:
error = err
if writer in self.writers: # remove this download attempt
self.writers.remove(writer)
if not error: # the blob downloaded, cancel all the other download attempts and set the result
while self.writers:
other = self.writers.pop()
other.finished.cancel()
t = self.loop.create_task(self.save_verified_blob(writer))
t.add_done_callback(lambda *_: self.finished_writing.set())
return
if isinstance(error, (InvalidBlobHashError, InvalidDataError)):
log.error("writer error downloading %s: %s", self.blob_hash[:8], str(error))
elif not isinstance(error, (DownloadCancelledError, asyncio.CancelledError, asyncio.TimeoutError)):
log.exception("something else")
raise error
return callback
async def save_verified_blob(self, writer):
def _save_verified():
# log.debug(f"write blob file {self.blob_hash[:8]} from {writer.peer.address}")
if not self.saved_verified_blob and not os.path.isfile(self.file_path):
if self.get_length() == len(writer.verified_bytes):
with open(self.file_path, 'wb') as write_handle:
write_handle.write(writer.verified_bytes)
self.saved_verified_blob = True
else:
raise Exception("length mismatch")
if self.verified.is_set():
return
async with self.blob_write_lock:
await self.loop.run_in_executor(None, _save_verified)
if self.blob_completed_callback:
await self.blob_completed_callback(self)
self.verified.set()
def open_for_writing(self) -> HashBlobWriter:
if os.path.exists(self.file_path):
raise OSError(f"File already exists '{self.file_path}'")
fut = asyncio.Future(loop=self.loop)
writer = HashBlobWriter(self.blob_hash, self.get_length, fut)
self.writers.append(writer)
fut.add_done_callback(self.writer_finished(writer))
return writer
async def sendfile(self, writer: asyncio.StreamWriter) -> int:
"""
open a blob file to be written by peer, supports concurrent
writers, as long as they are from differnt peers.
returns tuple of (writer, finished_deferred)
writer - a file like object with a write() function, close() when finished
finished_deferred - deferred that is fired when write is finished and returns
a instance of itself as HashBlob
Read and send the file to the writer and return the number of bytes sent
"""
if not peer in self.writers:
log.debug("Opening %s to be written by %s", str(self), str(peer))
finished_deferred = defer.Deferred()
writer = HashBlobWriter(self.get_length, self.writer_finished)
self.writers[peer] = (writer, finished_deferred)
return (writer, finished_deferred)
log.warning("Tried to download the same file twice simultaneously from the same peer")
return None, None
def open_for_reading(self):
"""
open blob for reading
with open(self.file_path, 'rb') as handle:
return await self.loop.sendfile(writer.transport, handle)
returns a file like object that can be read() from, and closed() when
finished
"""
if self._verified is True:
f = open(self.file_path, 'rb')
reader = HashBlobReader(f, self.reader_finished)
self.readers += 1
return reader
return None
async def close(self):
while self.writers:
self.writers.pop().finished.cancel()
def delete(self):
"""
delete blob file from file system, prevent deletion
if a blob is being read from or written to
returns a deferred that firesback when delete is completed
"""
if not self.writers and not self.readers:
self._verified = False
async def delete(self):
await self.close()
async with self.blob_write_lock:
self.saved_verified_blob = False
if os.path.isfile(self.file_path):
os.remove(self.file_path)
def delete_from_file_system():
if os.path.isfile(self.file_path):
os.remove(self.file_path)
d = threads.deferToThread(delete_from_file_system)
def log_error(err):
log.warning("An error occurred deleting %s: %s",
str(self.file_path), err.getErrorMessage())
return err
d.addErrback(log_error)
return d
else:
return defer.fail(Failure(
ValueError("File is currently being read or written and cannot be deleted")))
@property
def verified(self):
def decrypt(self, key: bytes, iv: bytes) -> bytes:
"""
Protect verified from being modified by other classes.
verified is True if a write to a blob has completed succesfully,
or a blob has been read to have the same length as specified
in init
Decrypt a BlobFile to plaintext bytes
"""
return self._verified
with open(self.file_path, "rb") as f:
buff = f.read()
if len(buff) != self.length:
raise ValueError("unexpected length")
cipher = Cipher(AES(key), modes.CBC(iv), backend=backend)
unpadder = PKCS7(AES.block_size).unpadder()
decryptor = cipher.decryptor()
return unpadder.update(decryptor.update(buff) + decryptor.finalize()) + unpadder.finalize()
@classmethod
async def create_from_unencrypted(cls, loop: asyncio.BaseEventLoop, blob_dir: str, key: bytes,
iv: bytes, unencrypted: bytes, blob_num: int) -> BlobInfo:
"""
Create an encrypted BlobFile from plaintext bytes
"""
blob_bytes, blob_hash = encrypt_blob_bytes(key, iv, unencrypted)
length = len(blob_bytes)
blob = cls(loop, blob_dir, blob_hash, length)
writer = blob.open_for_writing()
writer.write(blob_bytes)
await blob.verified.wait()
return BlobInfo(blob_num, length, binascii.hexlify(iv).decode(), blob_hash)
def set_length(self, length):
if self.length is not None and length == self.length:
return True
return
if self.length is None and 0 <= length <= MAX_BLOB_SIZE:
self.length = length
return True
log.warning("Got an invalid length. Previous length: %s, Invalid length: %s",
self.length, length)
return False
return
log.warning("Got an invalid length. Previous length: %s, Invalid length: %s", self.length, length)
def get_length(self):
return self.length
def get_is_verified(self):
return self.verified
def is_downloading(self):
if self.writers:
return True
return False
def reader_finished(self, reader):
self.readers -= 1
return defer.succeed(True)
def writer_finished(self, writer, err=None):
def fire_finished_deferred():
self._verified = True
for p, (w, finished_deferred) in self.writers.items():
if w == writer:
del self.writers[p]
finished_deferred.callback(self)
return True
log.warning(
"Somehow, the writer that was accepted as being valid was already removed: %s",
writer)
return False
def errback_finished_deferred(err):
for p, (w, finished_deferred) in self.writers.items():
if w == writer:
del self.writers[p]
finished_deferred.errback(err)
def cancel_other_downloads():
for p, (w, finished_deferred) in self.writers.items():
w.close()
if err is None:
if writer.len_so_far == self.length and writer.blob_hash == self.blob_hash:
if self._verified is False:
d = self.save_verified_blob(writer)
d.addCallbacks(lambda _: fire_finished_deferred(), errback_finished_deferred)
d.addCallback(lambda _: cancel_other_downloads())
else:
errback_finished_deferred(Failure(DownloadCanceledError()))
d = defer.succeed(True)
else:
err_string = "length vs expected: {0}, {1}, hash vs expected: {2}, {3}"
err_string = err_string.format(self.length, writer.len_so_far, self.blob_hash,
writer.blob_hash)
errback_finished_deferred(Failure(InvalidDataError(err_string)))
d = defer.succeed(True)
else:
errback_finished_deferred(err)
d = defer.succeed(True)
d.addBoth(lambda _: writer.close_handle())
return d
def save_verified_blob(self, writer):
# we cannot have multiple _save_verified_blob interrupting
# each other, can happen since startProducing is a deferred
return self.blob_write_lock.run(self._save_verified_blob, writer)
@defer.inlineCallbacks
def _save_verified_blob(self, writer):
if self.saved_verified_blob is False:
writer.write_handle.seek(0)
out_path = os.path.join(self.blob_dir, self.blob_hash)
producer = FileBodyProducer(writer.write_handle)
yield producer.startProducing(open(out_path, 'wb'))
self.saved_verified_blob = True
defer.returnValue(True)
else:
raise DownloadCanceledError()
return self.verified.is_set()
+19
View File
@@ -0,0 +1,19 @@
import typing
class BlobInfo:
def __init__(self, blob_num: int, length: int, iv: str, blob_hash: typing.Optional[str] = None):
self.blob_hash = blob_hash
self.blob_num = blob_num
self.length = length
self.iv = iv
def as_dict(self) -> typing.Dict:
d = {
'length': self.length,
'blob_num': self.blob_num,
'iv': self.iv,
}
if self.blob_hash: # non-terminator blobs have a blob hash
d['blob_hash'] = self.blob_hash
return d
+89
View File
@@ -0,0 +1,89 @@
import typing
import asyncio
import logging
from sqlite3 import IntegrityError
from lbrynet.extras.daemon.storage import SQLiteStorage
from lbrynet.blob.blob_file import BlobFile
from lbrynet.stream.descriptor import StreamDescriptor
if typing.TYPE_CHECKING:
from lbrynet.dht.protocol.data_store import DictDataStore
log = logging.getLogger(__name__)
class BlobFileManager:
def __init__(self, loop: asyncio.BaseEventLoop, blob_dir: str, storage: SQLiteStorage,
node_data_store: typing.Optional['DictDataStore'] = None):
"""
This class stores blobs on the hard disk
blob_dir - directory where blobs are stored
storage - SQLiteStorage object
"""
self.loop = loop
self.blob_dir = blob_dir
self.storage = storage
self._node_data_store = node_data_store
self.completed_blob_hashes: typing.Set[str] = set() if not self._node_data_store\
else self._node_data_store.completed_blobs
self.blobs: typing.Dict[str, BlobFile] = {}
async def setup(self) -> bool:
raw_blob_hashes = await self.get_all_verified_blobs()
self.completed_blob_hashes.update(raw_blob_hashes)
return True
def get_blob(self, blob_hash, length: typing.Optional[int] = None):
if blob_hash in self.blobs:
if length and self.blobs[blob_hash].length is None:
self.blobs[blob_hash].set_length(length)
else:
self.blobs[blob_hash] = BlobFile(self.loop, self.blob_dir, blob_hash, length, self.blob_completed)
return self.blobs[blob_hash]
def get_stream_descriptor(self, sd_hash):
return StreamDescriptor.from_stream_descriptor_blob(self.loop, self.blob_dir, self.get_blob(sd_hash))
async def blob_completed(self, blob: BlobFile):
if blob.blob_hash is None:
raise Exception("Blob hash is None")
if not blob.length:
raise Exception("Blob has a length of 0")
if blob.blob_hash not in self.completed_blob_hashes:
self.completed_blob_hashes.add(blob.blob_hash)
await self.storage.add_completed_blob(blob.blob_hash)
def check_completed_blobs(self, blob_hashes: typing.List[str]) -> typing.List[str]:
"""Returns of the blobhashes_to_check, which are valid"""
blobs = [self.get_blob(b) for b in blob_hashes]
return [blob.blob_hash for blob in blobs if blob.get_is_verified()]
async def set_should_announce(self, blob_hash: str, should_announce: bool):
now = self.loop.time()
return await self.storage.set_should_announce(blob_hash, now, should_announce)
async def get_all_verified_blobs(self) -> typing.List[str]:
blob_hashes = await self.storage.get_all_blob_hashes()
return self.check_completed_blobs(blob_hashes)
async def delete_blobs(self, blob_hashes: typing.List[str]):
bh_to_delete_from_db = []
for blob_hash in blob_hashes:
if not blob_hash:
continue
try:
blob = self.get_blob(blob_hash)
await blob.delete()
bh_to_delete_from_db.append(blob_hash)
except Exception as e:
log.warning("Failed to delete blob file. Reason: %s", e)
if blob_hash in self.completed_blob_hashes:
self.completed_blob_hashes.remove(blob_hash)
if blob_hash in self.blobs:
del self.blobs[blob_hash]
try:
await self.storage.delete_blobs_from_db(bh_to_delete_from_db)
except IntegrityError as err:
if str(err) != "FOREIGN KEY constraint failed":
raise err
-51
View File
@@ -1,51 +0,0 @@
import os
import logging
from io import BytesIO
from twisted.internet import defer
from twisted.web.client import FileBodyProducer
from lbrynet.core.cryptoutils import get_lbry_hash_obj
log = logging.getLogger(__name__)
class BlobFileCreator(object):
"""
This class is used to create blobs on the local filesystem
when we do not know the blob hash beforehand (i.e, when creating
a new stream)
"""
def __init__(self, blob_dir):
self.blob_dir = blob_dir
self.buffer = BytesIO()
self._is_open = True
self._hashsum = get_lbry_hash_obj()
self.len_so_far = 0
self.blob_hash = None
self.length = None
@defer.inlineCallbacks
def close(self):
self.length = self.len_so_far
self.blob_hash = self._hashsum.hexdigest()
if self.blob_hash and self._is_open and self.length > 0:
# do not save 0 length files (empty tail blob in streams)
# or if its been closed already
self.buffer.seek(0)
out_path = os.path.join(self.blob_dir, self.blob_hash)
producer = FileBodyProducer(self.buffer)
yield producer.startProducing(open(out_path, 'wb'))
self._is_open = False
if self.length > 0:
defer.returnValue(self.blob_hash)
else:
# 0 length files (empty tail blob in streams )
# must return None as their blob_hash for
# it to be saved properly by EncryptedFileMetadataManagers
defer.returnValue(None)
def write(self, data):
if not self._is_open:
raise IOError
self._hashsum.update(data)
self.len_so_far += len(data)
self.buffer.write(data)
-32
View File
@@ -1,32 +0,0 @@
import logging
log = logging.getLogger(__name__)
class HashBlobReader(object):
"""
This is a file like reader class that supports
read(size) and close()
"""
def __init__(self, read_handle, finished_cb):
self.finished_cb = finished_cb
self.finished_cb_d = None
self.read_handle = read_handle
def __del__(self):
if self.finished_cb_d is None:
log.warn("Garbage collection was called, but reader for %s was not closed yet",
self.read_handle.name)
self.close()
def read(self, size=-1):
return self.read_handle.read(size)
def close(self):
# if we've already closed and called finished_cb, do nothing
if self.finished_cb_d is not None:
return
self.read_handle.close()
self.finished_cb_d = self.finished_cb(self)
+52 -39
View File
@@ -1,58 +1,71 @@
import typing
import logging
import asyncio
from io import BytesIO
from twisted.python.failure import Failure
from lbrynet.core.Error import DownloadCanceledError, InvalidDataError
from lbrynet.core.cryptoutils import get_lbry_hash_obj
from lbrynet.error import InvalidBlobHashError, InvalidDataError
from lbrynet.cryptoutils import get_lbry_hash_obj
log = logging.getLogger(__name__)
class HashBlobWriter(object):
def __init__(self, length_getter, finished_cb):
self.write_handle = BytesIO()
self.length_getter = length_getter
self.finished_cb = finished_cb
self.finished_cb_d = None
class HashBlobWriter:
def __init__(self, expected_blob_hash: str, get_length: typing.Callable[[], int],
finished: asyncio.Future):
self.expected_blob_hash = expected_blob_hash
self.get_length = get_length
self.buffer = BytesIO()
self.finished = finished
self.finished.add_done_callback(lambda *_: self.close_handle())
self._hashsum = get_lbry_hash_obj()
self.len_so_far = 0
self.verified_bytes = b''
def __del__(self):
if self.finished_cb_d is None:
log.warn("Garbage collection was called, but writer was not closed yet")
self.close()
if self.buffer is not None:
log.warning("Garbage collection was called, but writer was not closed yet")
self.close_handle()
@property
def blob_hash(self):
def calculate_blob_hash(self) -> str:
return self._hashsum.hexdigest()
def write(self, data):
if self.write_handle is None:
log.exception("writer has already been closed")
def closed(self):
return self.buffer is None or self.buffer.closed
def write(self, data: bytes):
expected_length = self.get_length()
if not expected_length:
raise IOError("unknown blob length")
if self.buffer is None:
log.warning("writer has already been closed")
if not self.finished.done():
self.finished.cancel()
return
raise IOError('I/O operation on closed file')
self._hashsum.update(data)
self.len_so_far += len(data)
if self.len_so_far > self.length_getter():
self.finished_cb_d = self.finished_cb(
self,
Failure(InvalidDataError("Length so far is greater than the expected length."
" %s to %s" % (self.len_so_far,
self.length_getter()))))
else:
self.write_handle.write(data)
if self.len_so_far == self.length_getter():
self.finished_cb_d = self.finished_cb(self)
if self.len_so_far > expected_length:
self.close_handle()
self.finished.set_result(InvalidDataError(
f'Length so far is greater than the expected length. {self.len_so_far} to {expected_length}'
))
return
self.buffer.write(data)
if self.len_so_far == expected_length:
blob_hash = self.calculate_blob_hash()
if blob_hash != self.expected_blob_hash:
self.close_handle()
self.finished.set_result(InvalidBlobHashError(
f"blob hash is {blob_hash} vs expected {self.expected_blob_hash}"
))
return
self.buffer.seek(0)
self.verified_bytes = self.buffer.read()
self.close_handle()
if self.finished and not (self.finished.done() or self.finished.cancelled()):
self.finished.set_result(None)
def close_handle(self):
if self.write_handle is not None:
self.write_handle.close()
self.write_handle = None
def close(self, reason=None):
# if we've already called finished_cb because we either finished writing
# or closed already, do nothing
if self.finished_cb_d is not None:
return
if reason is None:
reason = Failure(DownloadCanceledError())
self.finished_cb_d = self.finished_cb(self, reason)
if self.buffer is not None:
self.buffer.close()
self.buffer = None
File renamed without changes.
+168
View File
@@ -0,0 +1,168 @@
import asyncio
import logging
import typing
from lbrynet.blob_exchange.serialization import BlobResponse, BlobRequest
if typing.TYPE_CHECKING:
from lbrynet.blob.blob_file import BlobFile
from lbrynet.blob.writer import HashBlobWriter
log = logging.getLogger(__name__)
class BlobExchangeClientProtocol(asyncio.Protocol):
def __init__(self, loop: asyncio.BaseEventLoop, peer_timeout: typing.Optional[float] = 10):
self.loop = loop
self.peer_port: typing.Optional[int] = None
self.peer_address: typing.Optional[str] = None
self.peer_timeout = peer_timeout
self.transport: asyncio.Transport = None
self.writer: 'HashBlobWriter' = None
self.blob: 'BlobFile' = None
self.download_running = asyncio.Event(loop=self.loop)
self._blob_bytes_received = 0
self._response_fut: asyncio.Future = None
self._request_lock = asyncio.Lock(loop=self.loop)
def handle_data_received(self, data: bytes):
if self.transport.is_closing():
if self._response_fut and not (self._response_fut.done() or self._response_fut.cancelled()):
self._response_fut.cancel()
return
response = BlobResponse.deserialize(data)
if response.responses and self.blob:
blob_response = response.get_blob_response()
if blob_response and not blob_response.error and blob_response.blob_hash == self.blob.blob_hash:
self.blob.set_length(blob_response.length)
elif blob_response and not blob_response.error and self.blob.blob_hash != blob_response.blob_hash:
log.warning("mismatch with self.blob %s", self.blob.blob_hash)
return
if response.responses:
self._response_fut.set_result(response)
if response.blob_data and self.writer and not self.writer.closed():
self._blob_bytes_received += len(response.blob_data)
try:
self.writer.write(response.blob_data)
except IOError as err:
log.error("error downloading blob: %s", err)
def data_received(self, data):
try:
return self.handle_data_received(data)
except (asyncio.CancelledError, asyncio.TimeoutError) as err:
if self._response_fut and not self._response_fut.done():
self._response_fut.set_exception(err)
async def _download_blob(self) -> typing.Tuple[bool, bool]:
request = BlobRequest.make_request_for_blob_hash(self.blob.blob_hash)
try:
self.transport.write(request.serialize())
response: BlobResponse = await asyncio.wait_for(self._response_fut, self.peer_timeout, loop=self.loop)
availability_response = response.get_availability_response()
price_response = response.get_price_response()
blob_response = response.get_blob_response()
if (not blob_response or blob_response.error) and\
(not availability_response or not availability_response.available_blobs):
log.warning("blob not in availability response")
return False, True
elif availability_response.available_blobs and \
availability_response.available_blobs != [self.blob.blob_hash]:
log.warning("blob availability response doesn't match our request")
return False, False
if not price_response or price_response.blob_data_payment_rate != 'RATE_ACCEPTED':
log.warning("data rate rejected")
return False, False
if not blob_response or blob_response.error:
log.warning("blob cant be downloaded from this peer")
return False, True
if not blob_response.error and blob_response.blob_hash != self.blob.blob_hash:
log.warning("incoming blob hash mismatch")
return False, False
if self.blob.length is not None and self.blob.length != blob_response.length:
log.warning("incoming blob unexpected length")
return False, False
msg = f"downloading {self.blob.blob_hash[:8]} from {self.peer_address}:{self.peer_port}," \
f" timeout in {self.peer_timeout}"
log.info(msg)
await asyncio.wait_for(self.writer.finished, self.peer_timeout, loop=self.loop)
await self.blob.finished_writing.wait()
msg = f"downloaded {self.blob.blob_hash[:8]} from {self.peer_address}"
log.info(msg)
return True, True
except asyncio.CancelledError:
return False, True
except asyncio.TimeoutError:
return False, False
finally:
await self.close()
async def close(self):
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
if self.writer and not self.writer.closed():
self.writer.close_handle()
if self.blob:
await self.blob.close()
self.download_running.clear()
self._response_fut = None
self.writer = None
self.blob = None
if self.transport:
self.transport.close()
self.transport = None
async def download_blob(self, blob: 'BlobFile') -> typing.Tuple[bool, bool]:
if blob.get_is_verified():
return False, True
async with self._request_lock:
try:
if self.download_running.is_set():
log.info("wait for download already running")
await self.download_running.wait()
self.blob, self.writer, self._blob_bytes_received = blob, blob.open_for_writing(), 0
self.download_running.set()
self._response_fut = asyncio.Future(loop=self.loop)
return await self._download_blob()
except OSError:
log.error("race happened")
# i'm not sure how to fix this race condition - jack
return False, True
except asyncio.TimeoutError:
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
return False, False
except asyncio.CancelledError:
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
return False, True
def connection_made(self, transport: asyncio.Transport):
self.transport = transport
self.peer_address, self.peer_port = self.transport.get_extra_info('peername')
# log.info("connection made to %s: %s", self.peer_address, transport)
def connection_lost(self, reason):
# log.info("connection lost to %s (reason: %s)", self.peer_address, reason)
self.transport = None
self.loop.create_task(self.close())
async def request_blob(loop: asyncio.BaseEventLoop, blob: 'BlobFile', protocol: 'BlobExchangeClientProtocol',
address: str, tcp_port: int, peer_connect_timeout: float) -> typing.Tuple[bool, bool]:
"""
Returns [<downloaded blob>, <keep connection>]
"""
if blob.get_is_verified():
return False, True
if blob.get_is_verified():
log.info("already verified")
return False, True
try:
await asyncio.wait_for(loop.create_connection(lambda: protocol, address, tcp_port),
peer_connect_timeout, loop=loop)
return await protocol.download_blob(blob)
except (asyncio.TimeoutError, asyncio.CancelledError, ConnectionRefusedError, ConnectionAbortedError, OSError):
return False, False
+132
View File
@@ -0,0 +1,132 @@
import asyncio
import typing
import logging
from lbrynet import conf
from lbrynet.utils import drain_tasks
from lbrynet.blob_exchange.client import BlobExchangeClientProtocol, request_blob
if typing.TYPE_CHECKING:
from lbrynet.dht.node import Node
from lbrynet.dht.peer import KademliaPeer
from lbrynet.blob.blob_manager import BlobFileManager
from lbrynet.blob.blob_file import BlobFile
log = logging.getLogger(__name__)
def drain_into(a: list, b: list):
while a:
b.append(a.pop())
class BlobDownloader: # TODO: refactor to be the base class used by StreamDownloader
"""A single blob downloader"""
def __init__(self, loop: asyncio.BaseEventLoop, blob_manager: 'BlobFileManager', config: conf.Config):
self.loop = loop
self.blob_manager = blob_manager
self.new_peer_event = asyncio.Event(loop=self.loop)
self.active_connections: typing.Dict['KademliaPeer', BlobExchangeClientProtocol] = {}
self.running_download_requests: typing.List[asyncio.Task] = []
self.requested_from: typing.Dict[str, typing.Dict['KademliaPeer', asyncio.Task]] = {}
self.lock = asyncio.Lock(loop=self.loop)
self.blob: 'BlobFile' = None
self.blob_queue = asyncio.Queue(loop=self.loop)
self.blob_download_timeout = config.get('blob_download_timeout')
self.peer_connect_timeout = config.get('peer_connect_timeout')
self.max_connections = config.get('max_connections_per_stream')
async def _request_blob(self, peer: 'KademliaPeer'):
if self.blob.get_is_verified():
log.info("already verified")
return
if peer not in self.active_connections:
log.warning("not active, adding: %s", str(peer))
self.active_connections[peer] = BlobExchangeClientProtocol(self.loop, self.blob_download_timeout)
protocol = self.active_connections[peer]
success, keep_connection = await request_blob(self.loop, self.blob, protocol, peer.address, peer.tcp_port,
self.peer_connect_timeout)
await protocol.close()
if not keep_connection:
log.info("drop peer %s:%i", peer.address, peer.tcp_port)
if peer in self.active_connections:
async with self.lock:
del self.active_connections[peer]
return
log.info("keep peer %s:%i", peer.address, peer.tcp_port)
def _update_requests(self):
self.new_peer_event.clear()
if self.blob.blob_hash not in self.requested_from:
self.requested_from[self.blob.blob_hash] = {}
to_add = []
for peer in self.active_connections.keys():
if peer not in self.requested_from[self.blob.blob_hash] and peer not in to_add:
to_add.append(peer)
if to_add or self.running_download_requests:
log.info("adding download probes for %i peers to %i already active",
min(len(to_add), 8 - len(self.running_download_requests)),
len(self.running_download_requests))
else:
log.info("downloader idle...")
for peer in to_add:
if len(self.running_download_requests) >= 8:
break
task = self.loop.create_task(self._request_blob(peer))
self.requested_from[self.blob.blob_hash][peer] = task
self.running_download_requests.append(task)
def _add_peer_protocols(self, peers: typing.List['KademliaPeer']):
added = 0
for peer in peers:
if peer not in self.active_connections:
self.active_connections[peer] = BlobExchangeClientProtocol(self.loop, self.blob_download_timeout)
added += 1
if added:
if not self.new_peer_event.is_set():
log.info("added %i new peers", len(peers))
self.new_peer_event.set()
async def _accumulate_connections(self, node: 'Node'):
try:
async with node.stream_peer_search_junction(self.blob_queue) as search_junction:
async for peers in search_junction:
if not isinstance(peers, list): # TODO: what's up with this?
log.error("not a list: %s %s", peers, str(type(peers)))
else:
self._add_peer_protocols(peers)
return
except asyncio.CancelledError:
pass
async def get_blob(self, blob_hash: str, node: 'Node') -> 'BlobFile':
self.blob = self.blob_manager.get_blob(blob_hash)
if self.blob.get_is_verified():
return self.blob
accumulator = self.loop.create_task(self._accumulate_connections(node))
self.blob_queue.put_nowait(blob_hash)
try:
while not self.blob.get_is_verified():
if len(self.running_download_requests) < self.max_connections:
self._update_requests()
# drain the tasks into a temporary list
download_tasks = []
drain_into(self.running_download_requests, download_tasks)
got_new_peer = self.loop.create_task(self.new_peer_event.wait())
# wait for a new peer to be added or for a download attempt to finish
await asyncio.wait([got_new_peer] + download_tasks, return_when='FIRST_COMPLETED',
loop=self.loop)
if got_new_peer and not got_new_peer.done():
got_new_peer.cancel()
if self.blob.get_is_verified():
if got_new_peer and not got_new_peer.done():
got_new_peer.cancel()
drain_tasks(download_tasks)
return self.blob
except asyncio.CancelledError:
drain_tasks(self.running_download_requests)
raise
finally:
if accumulator and not accumulator.done():
accumulator.cancel()
+289
View File
@@ -0,0 +1,289 @@
import typing
import json
import logging
log = logging.getLogger(__name__)
class BlobMessage:
key = ''
def to_dict(self) -> typing.Dict:
raise NotImplementedError()
class BlobPriceRequest(BlobMessage):
key = 'blob_data_payment_rate'
def __init__(self, blob_data_payment_rate: float, **kwargs) -> None:
self.blob_data_payment_rate = blob_data_payment_rate
def to_dict(self) -> typing.Dict:
return {
self.key: self.blob_data_payment_rate
}
class BlobPriceResponse(BlobMessage):
key = 'blob_data_payment_rate'
rate_accepted = 'RATE_ACCEPTED'
rate_too_low = 'RATE_TOO_LOW'
rate_unset = 'RATE_UNSET'
def __init__(self, blob_data_payment_rate: str, **kwargs) -> None:
if blob_data_payment_rate not in (self.rate_accepted, self.rate_too_low, self.rate_unset):
raise ValueError(blob_data_payment_rate)
self.blob_data_payment_rate = blob_data_payment_rate
def to_dict(self) -> typing.Dict:
return {
self.key: self.blob_data_payment_rate
}
class BlobAvailabilityRequest(BlobMessage):
key = 'requested_blobs'
def __init__(self, requested_blobs: typing.List[str], lbrycrd_address: typing.Optional[bool] = True,
**kwargs) -> None:
assert len(requested_blobs)
self.requested_blobs = requested_blobs
self.lbrycrd_address = lbrycrd_address
def to_dict(self) -> typing.Dict:
return {
self.key: self.requested_blobs,
'lbrycrd_address': self.lbrycrd_address
}
class BlobAvailabilityResponse(BlobMessage):
key = 'available_blobs'
def __init__(self, available_blobs: typing.List[str], lbrycrd_address: typing.Optional[str] = True,
**kwargs) -> None:
self.available_blobs = available_blobs
self.lbrycrd_address = lbrycrd_address
def to_dict(self) -> typing.Dict:
d = {
self.key: self.available_blobs
}
if self.lbrycrd_address:
d['lbrycrd_address'] = self.lbrycrd_address
return d
class BlobDownloadRequest(BlobMessage):
key = 'requested_blob'
def __init__(self, requested_blob: str, **kwargs) -> None:
self.requested_blob = requested_blob
def to_dict(self) -> typing.Dict:
return {
self.key: self.requested_blob
}
class BlobDownloadResponse(BlobMessage):
key = 'incoming_blob'
def __init__(self, **response: typing.Dict) -> None:
incoming_blob = response[self.key]
self.error = None
self.incoming_blob = None
if 'error' in incoming_blob:
self.error = incoming_blob['error']
else:
self.incoming_blob = {'blob_hash': incoming_blob['blob_hash'], 'length': incoming_blob['length']}
self.length = None if not self.incoming_blob else self.incoming_blob['length']
self.blob_hash = None if not self.incoming_blob else self.incoming_blob['blob_hash']
def to_dict(self) -> typing.Dict:
return {
self.key if not self.error else 'error': self.incoming_blob or self.error,
}
class BlobPaymentAddressRequest(BlobMessage):
key = 'lbrycrd_address'
def __init__(self, lbrycrd_address: str, **kwargs) -> None:
self.lbrycrd_address = lbrycrd_address
def to_dict(self) -> typing.Dict:
return {
self.key: self.lbrycrd_address
}
class BlobPaymentAddressResponse(BlobPaymentAddressRequest):
pass
class BlobErrorResponse(BlobMessage):
key = 'error'
def __init__(self, error: str, **kwargs) -> None:
self.error = error
def to_dict(self) -> typing.Dict:
return {
self.key: self.error
}
blob_request_types = typing.Union[BlobPriceRequest, BlobAvailabilityRequest, BlobDownloadRequest,
BlobPaymentAddressRequest]
blob_response_types = typing.Union[BlobPriceResponse, BlobAvailabilityResponse, BlobDownloadResponse,
BlobErrorResponse, BlobPaymentAddressResponse]
def _parse_blob_response(response_msg: bytes) -> typing.Tuple[typing.Optional[typing.Dict], bytes]:
# scenarios:
# <json>
# <blob bytes>
# <json><blob bytes>
extra_data = b''
response = None
curr_pos = 0
while True:
next_close_paren = response_msg.find(b'}', curr_pos)
if next_close_paren == -1:
break
curr_pos = next_close_paren + 1
try:
response = json.loads(response_msg[:curr_pos])
if not isinstance(response, dict):
raise ValueError()
for key in response.keys():
if key not in [
BlobPaymentAddressResponse.key,
BlobAvailabilityResponse.key,
BlobPriceResponse.key,
BlobDownloadResponse.key]:
raise ValueError()
extra_data = response_msg[curr_pos:]
break
except ValueError:
response = None
if response is None:
extra_data = response_msg
return response, extra_data
class BlobRequest:
def __init__(self, requests: typing.List[blob_request_types]) -> None:
self.requests = requests
def to_dict(self):
d = {}
for request in self.requests:
d.update(request.to_dict())
return d
def _get_request(self, request_type: blob_request_types):
request = tuple(filter(lambda r: type(r) == request_type, self.requests))
if request:
return request[0]
def get_availability_request(self) -> typing.Optional[BlobAvailabilityRequest]:
response = self._get_request(BlobAvailabilityRequest)
if response:
return response
def get_price_request(self) -> typing.Optional[BlobPriceRequest]:
response = self._get_request(BlobPriceRequest)
if response:
return response
def get_blob_request(self) -> typing.Optional[BlobDownloadRequest]:
response = self._get_request(BlobDownloadRequest)
if response:
return response
def get_address_request(self) -> typing.Optional[BlobPaymentAddressRequest]:
response = self._get_request(BlobPaymentAddressRequest)
if response:
return response
def serialize(self) -> bytes:
return json.dumps(self.to_dict()).encode()
@classmethod
def deserialize(cls, data: bytes) -> 'BlobRequest':
request = json.loads(data)
return cls([
request_type(**request)
for request_type in (BlobPriceRequest, BlobAvailabilityRequest, BlobDownloadRequest,
BlobPaymentAddressRequest)
if request_type.key in request
])
@classmethod
def make_request_for_blob_hash(cls, blob_hash: str) -> 'BlobRequest':
return cls(
[BlobAvailabilityRequest([blob_hash]), BlobPriceRequest(0.0), BlobDownloadRequest(blob_hash)]
)
class BlobResponse:
def __init__(self, responses: typing.List[blob_response_types], blob_data: typing.Optional[bytes] = None) -> None:
self.responses = responses
self.blob_data = blob_data
def to_dict(self):
d = {}
for response in self.responses:
d.update(response.to_dict())
return d
def _get_response(self, response_type: blob_response_types):
response = tuple(filter(lambda r: type(r) == response_type, self.responses))
if response:
return response[0]
def get_error_response(self) -> typing.Optional[BlobErrorResponse]:
error = self._get_response(BlobErrorResponse)
if error:
log.error(error)
return error
def get_availability_response(self) -> typing.Optional[BlobAvailabilityResponse]:
response = self._get_response(BlobAvailabilityResponse)
if response:
return response
def get_price_response(self) -> typing.Optional[BlobPriceResponse]:
response = self._get_response(BlobPriceResponse)
if response:
return response
def get_blob_response(self) -> typing.Optional[BlobDownloadResponse]:
response = self._get_response(BlobDownloadResponse)
if response:
return response
def get_address_response(self) -> typing.Optional[BlobPaymentAddressResponse]:
response = self._get_response(BlobPaymentAddressResponse)
if response:
return response
def serialize(self) -> bytes:
return json.dumps(self.to_dict()).encode()
@classmethod
def deserialize(cls, data: bytes) -> 'BlobResponse':
response, extra = _parse_blob_response(data)
requests = []
if response:
requests.extend([
response_type(**response)
for response_type in (BlobPriceResponse, BlobAvailabilityResponse, BlobDownloadResponse,
BlobErrorResponse, BlobPaymentAddressResponse)
if response_type.key in response
])
return cls(requests, extra)
+119
View File
@@ -0,0 +1,119 @@
import asyncio
import binascii
import logging
import typing
from json.decoder import JSONDecodeError
from lbrynet.blob_exchange.serialization import BlobResponse, BlobRequest, blob_response_types
from lbrynet.blob_exchange.serialization import BlobAvailabilityResponse, BlobPriceResponse, BlobDownloadResponse, \
BlobPaymentAddressResponse
if typing.TYPE_CHECKING:
from lbrynet.blob.blob_manager import BlobFileManager
log = logging.getLogger(__name__)
class BlobServerProtocol(asyncio.Protocol):
def __init__(self, loop: asyncio.BaseEventLoop, blob_manager: 'BlobFileManager', lbrycrd_address: str):
self.loop = loop
self.blob_manager = blob_manager
self.server_task: asyncio.Task = None
self.started_listening = asyncio.Event(loop=self.loop)
self.buf = b''
self.transport = None
self.lbrycrd_address = lbrycrd_address
def connection_made(self, transport):
self.transport = transport
def send_response(self, responses: typing.List[blob_response_types]):
to_send = []
while responses:
to_send.append(responses.pop())
self.transport.write(BlobResponse(to_send).serialize())
async def handle_request(self, request: BlobRequest):
addr = self.transport.get_extra_info('peername')
peer_address, peer_port = addr
responses = []
address_request = request.get_address_request()
if address_request:
responses.append(BlobPaymentAddressResponse(lbrycrd_address=self.lbrycrd_address))
availability_request = request.get_availability_request()
if availability_request:
responses.append(BlobAvailabilityResponse(available_blobs=list(set((
filter(lambda blob_hash: blob_hash in self.blob_manager.completed_blob_hashes,
availability_request.requested_blobs)
)))))
price_request = request.get_price_request()
if price_request:
responses.append(BlobPriceResponse(blob_data_payment_rate='RATE_ACCEPTED'))
download_request = request.get_blob_request()
if download_request:
blob = self.blob_manager.get_blob(download_request.requested_blob)
if blob.get_is_verified():
incoming_blob = {'blob_hash': blob.blob_hash, 'length': blob.length}
responses.append(BlobDownloadResponse(incoming_blob=incoming_blob))
self.send_response(responses)
log.info("send %s to %s:%i", blob.blob_hash[:8], peer_address, peer_port)
sent = await blob.sendfile(self)
log.info("sent %s (%i bytes) to %s:%i", blob.blob_hash[:8], sent, peer_address, peer_port)
if responses:
self.send_response(responses)
# self.transport.close()
def data_received(self, data):
request = None
if data:
message, separator, remainder = data.rpartition(b'}')
if not separator:
self.buf += data
return
try:
request = BlobRequest.deserialize(data)
self.buf = remainder
except JSONDecodeError:
addr = self.transport.get_extra_info('peername')
peer_address, peer_port = addr
log.error("failed to decode blob request from %s:%i (%i bytes): %s", peer_address, peer_port,
len(data), '' if not data else binascii.hexlify(data).decode())
if not request:
addr = self.transport.get_extra_info('peername')
peer_address, peer_port = addr
log.warning("failed to decode blob request from %s:%i", peer_address, peer_port)
self.transport.close()
return
self.loop.create_task(self.handle_request(request))
class BlobServer:
def __init__(self, loop: asyncio.BaseEventLoop, blob_manager: 'BlobFileManager', lbrycrd_address: str):
self.loop = loop
self.blob_manager = blob_manager
self.server_task: asyncio.Task = None
self.started_listening = asyncio.Event(loop=self.loop)
self.lbrycrd_address = lbrycrd_address
def start_server(self, port: int, interface: typing.Optional[str] = '0.0.0.0'):
if self.server_task is not None:
raise Exception("already running")
async def _start_server():
server = await self.loop.create_server(
lambda: BlobServerProtocol(self.loop, self.blob_manager, self.lbrycrd_address),
interface, port
)
self.started_listening.set()
log.info("Blob server listening on TCP %s:%i", interface, port)
async with server:
await server.serve_forever()
self.server_task = self.loop.create_task(_start_server())
def stop_server(self):
if self.server_task:
self.server_task.cancel()
self.server_task = None
log.info("Stopped blob server")
+1 -1
View File
@@ -1,2 +1,2 @@
# dont touch this. CI server changes this during build/deployment
# don't touch this. CI server changes this during build/deployment
BUILD = "dev"
+560 -591
View File
File diff suppressed because it is too large. Load diff
-111
View File
@@ -1,111 +0,0 @@
import logging
import random
import time
from twisted.internet import defer
from twisted.internet.task import LoopingCall
from decimal import Decimal
log = logging.getLogger(__name__)
class BlobAvailabilityTracker(object):
"""
Class to track peer counts for known blobs, and to discover new popular blobs
Attributes:
availability (dict): dictionary of peers for known blobs
"""
def __init__(self, blob_manager, peer_finder, dht_node):
self.availability = {}
self._last_mean_availability = Decimal(0.0)
self._blob_manager = blob_manager
self._peer_finder = peer_finder
self._dht_node = dht_node
self._check_popular = LoopingCall(self._update_most_popular)
self._check_mine = LoopingCall(self._update_mine)
def start(self):
log.info("Starting blob availability tracker.")
self._check_popular.start(600)
self._check_mine.start(600)
def stop(self):
log.info("Stopping blob availability tracker.")
if self._check_popular.running:
self._check_popular.stop()
if self._check_mine.running:
self._check_mine.stop()
def get_blob_availability(self, blob, timeout=None):
def _get_peer_count(peers):
have_blob = sum(1 for peer in peers if peer.is_available())
return {blob: have_blob}
d = self._peer_finder.find_peers_for_blob(blob, timeout)
d.addCallback(_get_peer_count)
return d
def get_availability_for_blobs(self, blobs, timeout=None):
dl = [self.get_blob_availability(blob, timeout) for blob in blobs if blob]
d = defer.DeferredList(dl)
d.addCallback(lambda results: [val for success, val in results if success])
return d
@property
def last_mean_availability(self):
return max(Decimal(0.01), self._last_mean_availability)
def _update_peers_for_blob(self, blob):
def _save_peer_info(blob_hash, peers):
v = {blob_hash: peers}
self.availability.update(v)
return v
d = self._peer_finder.find_peers_for_blob(blob)
d.addCallback(lambda r: [[c.host, c.port, c.is_available()] for c in r])
d.addCallback(lambda peers: _save_peer_info(blob, peers))
return d
def _get_most_popular(self):
dl = []
for (hash, _) in self._dht_node.get_most_popular_hashes(10):
encoded = hash.encode('hex')
dl.append(self._update_peers_for_blob(encoded))
return defer.DeferredList(dl)
def _update_most_popular(self):
d = self._get_most_popular()
d.addCallback(lambda _: self._set_mean_peers())
def _update_mine(self):
def _get_peers(blobs):
dl = []
for hash in blobs:
dl.append(self._update_peers_for_blob(hash))
return defer.DeferredList(dl)
def sample(blobs):
return random.sample(blobs, min(len(blobs), 10))
start = time.time()
log.debug('==> Updating the peers for my blobs')
d = self._blob_manager.get_all_verified_blobs()
# as far as I can tell, this only is used to set _last_mean_availability
# which... seems like a very expensive operation for such little payoff.
# so taking a sample should get about the same effect as querying the entire
# list of blobs
d.addCallback(sample)
d.addCallback(_get_peers)
d.addCallback(lambda _: self._set_mean_peers())
d.addCallback(lambda _: log.debug('<== Done updating peers for my blobs. Took %s seconds',
time.time() - start))
# although unused, need to return or else the looping call
# could overrun on a previous call
return d
def _set_mean_peers(self):
num_peers = [len(self.availability[blob]) for blob in self.availability]
mean = Decimal(sum(num_peers)) / Decimal(max(1, len(num_peers)))
self._last_mean_availability = mean
-19
View File
@@ -1,19 +0,0 @@
class BlobInfo(object):
"""
This structure is used to represent the metadata of a blob.
@ivar blob_hash: The sha384 hashsum of the blob's data.
@type blob_hash: string, hex-encoded
@ivar blob_num: For streams, the position of the blob in the stream.
@type blob_num: integer
@ivar length: The length of the blob in bytes.
@type length: integer
"""
def __init__(self, blob_hash, blob_num, length):
self.blob_hash = blob_hash
self.blob_num = blob_num
self.length = length
-162
View File
@@ -1,162 +0,0 @@
import logging
import os
from sqlite3 import IntegrityError
from twisted.internet import threads, defer, reactor, task
from lbrynet import conf
from lbrynet.blob.blob_file import BlobFile
from lbrynet.blob.creator import BlobFileCreator
from lbrynet.core.server.DHTHashAnnouncer import DHTHashSupplier
log = logging.getLogger(__name__)
class DiskBlobManager(DHTHashSupplier):
def __init__(self, hash_announcer, blob_dir, storage):
"""
This class stores blobs on the hard disk,
blob_dir - directory where blobs are stored
db_dir - directory where sqlite database of blob information is stored
"""
DHTHashSupplier.__init__(self, hash_announcer)
self.storage = storage
self.announce_head_blobs_only = conf.settings['announce_head_blobs_only']
self.blob_dir = blob_dir
self.blob_creator_type = BlobFileCreator
# TODO: consider using an LRU for blobs as there could potentially
# be thousands of blobs loaded up, many stale
self.blobs = {}
self.blob_hashes_to_delete = {} # {blob_hash: being_deleted (True/False)}
self.check_should_announce_lc = None
if conf.settings['run_reflector_server']:
self.check_should_announce_lc = task.LoopingCall(self.storage.verify_will_announce_all_head_and_sd_blobs)
def setup(self):
if self.check_should_announce_lc and not self.check_should_announce_lc.running:
self.check_should_announce_lc.start(600)
return defer.succeed(True)
def stop(self):
if self.check_should_announce_lc and self.check_should_announce_lc.running:
self.check_should_announce_lc.stop()
return defer.succeed(True)
def get_blob(self, blob_hash, length=None):
"""Return a blob identified by blob_hash, which may be a new blob or a
blob that is already on the hard disk
"""
if length is not None and not isinstance(length, int):
raise Exception("invalid length type: %s (%s)" % (length, str(type(length))))
if blob_hash in self.blobs:
return defer.succeed(self.blobs[blob_hash])
return self._make_new_blob(blob_hash, length)
def get_blob_creator(self):
return self.blob_creator_type(self.blob_dir)
def _make_new_blob(self, blob_hash, length=None):
log.debug('Making a new blob for %s', blob_hash)
blob = BlobFile(self.blob_dir, blob_hash, length)
self.blobs[blob_hash] = blob
return defer.succeed(blob)
def immediate_announce(self, blob_hashes):
if self.hash_announcer:
return self.hash_announcer.immediate_announce(blob_hashes)
raise Exception("Hash announcer not set")
@defer.inlineCallbacks
def blob_completed(self, blob, next_announce_time=None, should_announce=True):
if next_announce_time is None:
next_announce_time = self.get_next_announce_time()
yield self.storage.add_completed_blob(
blob.blob_hash, blob.length, next_announce_time, should_announce
)
# we announce all blobs immediately, if announce_head_blob_only is False
# otherwise, announce only if marked as should_announce
if not self.announce_head_blobs_only or should_announce:
reactor.callLater(0, self.immediate_announce, [blob.blob_hash])
def completed_blobs(self, blobhashes_to_check):
return self._completed_blobs(blobhashes_to_check)
def hashes_to_announce(self):
return self.storage.get_blobs_to_announce(self.hash_announcer)
def count_should_announce_blobs(self):
return self.storage.count_should_announce_blobs()
def set_should_announce(self, blob_hash, should_announce):
if blob_hash in self.blobs:
blob = self.blobs[blob_hash]
if blob.get_is_verified():
return self.storage.set_should_announce(
blob_hash, self.get_next_announce_time(), should_announce
)
return defer.succeed(False)
def get_should_announce(self, blob_hash):
return self.storage.should_announce(blob_hash)
def creator_finished(self, blob_creator, should_announce):
log.debug("blob_creator.blob_hash: %s", blob_creator.blob_hash)
if blob_creator.blob_hash is None:
raise Exception("Blob hash is None")
if blob_creator.blob_hash in self.blobs:
raise Exception("Creator finished for blob that is already marked as completed")
if blob_creator.length is None:
raise Exception("Blob has a length of 0")
new_blob = BlobFile(self.blob_dir, blob_creator.blob_hash, blob_creator.length)
self.blobs[blob_creator.blob_hash] = new_blob
next_announce_time = self.get_next_announce_time()
return self.blob_completed(new_blob, next_announce_time, should_announce)
def immediate_announce_all_blobs(self):
d = self._get_all_verified_blob_hashes()
d.addCallback(self.immediate_announce)
return d
def get_all_verified_blobs(self):
d = self._get_all_verified_blob_hashes()
d.addCallback(self.completed_blobs)
return d
@defer.inlineCallbacks
def delete_blobs(self, blob_hashes):
bh_to_delete_from_db = []
for blob_hash in blob_hashes:
try:
blob = yield self.get_blob(blob_hash)
yield blob.delete()
bh_to_delete_from_db.append(blob_hash)
del self.blobs[blob_hash]
except Exception as e:
log.warning("Failed to delete blob file. Reason: %s", e)
try:
yield self.storage.delete_blobs_from_db(bh_to_delete_from_db)
except IntegrityError as err:
if err.message != "FOREIGN KEY constraint failed":
raise err
@defer.inlineCallbacks
def _completed_blobs(self, blobhashes_to_check):
"""Returns of the blobhashes_to_check, which are valid"""
blobs = yield defer.DeferredList([self.get_blob(b) for b in blobhashes_to_check])
blob_hashes = [b.blob_hash for success, b in blobs if success and b.verified]
defer.returnValue(blob_hashes)
def _get_all_verified_blob_hashes(self):
d = self.storage.get_all_blob_hashes()
def get_verified_blobs(blobs):
verified_blobs = []
for blob_hash in blobs:
file_path = os.path.join(self.blob_dir, blob_hash)
if os.path.isfile(file_path):
verified_blobs.append(blob_hash)
return verified_blobs
d.addCallback(lambda blobs: threads.deferToThread(get_verified_blobs, blobs))
return d
-21
View File
@@ -1,21 +0,0 @@
class DownloadOptionChoice(object):
"""A possible choice that can be picked for some option.
An option can have one or more choices that can be picked from.
"""
def __init__(self, value, short_description, long_description, bool_options_description=None):
self.value = value
self.short_description = short_description
self.long_description = long_description
self.bool_options_description = bool_options_description
class DownloadOption(object):
"""An option for a user to select a value from several different choices."""
def __init__(self, option_types, long_description, short_description, default_value,
default_value_description):
self.option_types = option_types
self.long_description = long_description
self.short_description = short_description
self.default_value = default_value
self.default_value_description = default_value_description
-18
View File
@@ -1,18 +0,0 @@
class DummyHashAnnouncer(object):
def __init__(self, *args):
pass
def run_manage_loop(self):
pass
def stop(self):
pass
def add_supplier(self, *args):
pass
def hash_queue_size(self):
return 0
def immediate_announce(self, *args):
pass
-62
View File
@@ -1,62 +0,0 @@
from decimal import Decimal
class Offer(object):
"""A rate offer to download blobs from a host."""
RATE_ACCEPTED = "RATE_ACCEPTED"
RATE_TOO_LOW = "RATE_TOO_LOW"
RATE_UNSET = "RATE_UNSET"
def __init__(self, offer):
self._state = None
self.rate = None
if isinstance(offer, Decimal):
self.rate = round(offer, 5)
elif isinstance(offer, float):
self.rate = round(Decimal(offer), 5)
if self.rate is None or self.rate < Decimal(0.0):
self.unset()
@property
def is_accepted(self):
return self._state is Offer.RATE_ACCEPTED
@property
def is_too_low(self):
return self._state is Offer.RATE_TOO_LOW
@property
def is_unset(self):
return self._state is Offer.RATE_UNSET
@property
def message(self):
if self.is_accepted:
return Offer.RATE_ACCEPTED
elif self.is_too_low:
return Offer.RATE_TOO_LOW
elif self.is_unset:
return Offer.RATE_UNSET
return None
def accept(self):
if self.is_unset or self._state is None:
self._state = Offer.RATE_ACCEPTED
def reject(self):
if self.is_unset or self._state is None:
self._state = Offer.RATE_TOO_LOW
def unset(self):
self._state = Offer.RATE_UNSET
def handle(self, reply_message):
if reply_message == Offer.RATE_TOO_LOW:
self.reject()
elif reply_message == Offer.RATE_ACCEPTED:
self.accept()
elif reply_message == Offer.RATE_UNSET:
self.unset()
else:
raise Exception("Unknown offer reply %s" % str(reply_message))
-330
View File
@@ -1,330 +0,0 @@
from collections import defaultdict
import logging
import os
import unqlite
import time
from Crypto.Hash import SHA512
from Crypto.PublicKey import RSA
from lbrynet.core.client.ClientRequest import ClientRequest
from lbrynet.core.Error import RequestCanceledError
from lbrynet.interfaces import IRequestCreator, IQueryHandlerFactory, IQueryHandler, IWallet
from lbrynet.pointtraderclient import pointtraderclient
from twisted.internet import defer, threads
from zope.interface import implements
from twisted.python.failure import Failure
from lbrynet.core.Wallet import ReservedPoints
log = logging.getLogger(__name__)
class PTCWallet(object):
"""This class sends payments to peers and also ensures that expected payments are received.
This class is only intended to be used for testing."""
implements(IWallet)
def __init__(self, db_dir):
self.db_dir = db_dir
self.db = None
self.private_key = None
self.encoded_public_key = None
self.peer_pub_keys = {}
self.queued_payments = defaultdict(int)
self.expected_payments = defaultdict(list)
self.received_payments = defaultdict(list)
self.next_manage_call = None
self.payment_check_window = 3 * 60 # 3 minutes
self.new_payments_expected_time = time.time() - self.payment_check_window
self.known_transactions = []
self.total_reserved_points = 0.0
self.wallet_balance = 0.0
def manage(self):
"""Send payments, ensure expected payments are received"""
from twisted.internet import reactor
if time.time() < self.new_payments_expected_time + self.payment_check_window:
d1 = self._get_new_payments()
else:
d1 = defer.succeed(None)
d1.addCallback(lambda _: self._check_good_standing())
d2 = self._send_queued_points()
self.next_manage_call = reactor.callLater(60, self.manage)
dl = defer.DeferredList([d1, d2])
dl.addCallback(lambda _: self.get_balance())
def set_balance(balance):
self.wallet_balance = balance
dl.addCallback(set_balance)
return dl
def stop(self):
if self.next_manage_call is not None:
self.next_manage_call.cancel()
self.next_manage_call = None
d = self.manage()
self.next_manage_call.cancel()
self.next_manage_call = None
self.db = None
return d
def start(self):
def save_key(success, private_key):
if success is True:
self._save_private_key(private_key.exportKey())
return True
return False
def register_private_key(private_key):
self.private_key = private_key
self.encoded_public_key = self.private_key.publickey().exportKey()
d_r = pointtraderclient.register_new_account(private_key)
d_r.addCallback(save_key, private_key)
return d_r
def ensure_private_key_exists(encoded_private_key):
if encoded_private_key is not None:
self.private_key = RSA.importKey(encoded_private_key)
self.encoded_public_key = self.private_key.publickey().exportKey()
return True
else:
create_d = threads.deferToThread(RSA.generate, 4096)
create_d.addCallback(register_private_key)
return create_d
def start_manage():
self.manage()
return True
d = self._open_db()
d.addCallback(lambda _: self._get_wallet_private_key())
d.addCallback(ensure_private_key_exists)
d.addCallback(lambda _: start_manage())
return d
def get_info_exchanger(self):
return PointTraderKeyExchanger(self)
def get_wallet_info_query_handler_factory(self):
return PointTraderKeyQueryHandlerFactory(self)
def reserve_points(self, peer, amount):
"""Ensure a certain amount of points are available to be sent as
payment, before the service is rendered
@param peer: The peer to which the payment will ultimately be sent
@param amount: The amount of points to reserve
@return: A ReservedPoints object which is given to send_points
once the service has been rendered
"""
if self.wallet_balance >= self.total_reserved_points + amount:
self.total_reserved_points += amount
return ReservedPoints(peer, amount)
return None
def cancel_point_reservation(self, reserved_points):
"""
Return all of the points that were reserved previously for some ReservedPoints object
@param reserved_points: ReservedPoints previously returned by reserve_points
@return: None
"""
self.total_reserved_points -= reserved_points.amount
def send_points(self, reserved_points, amount):
"""
Schedule a payment to be sent to a peer
@param reserved_points: ReservedPoints object previously returned by reserve_points
@param amount: amount of points to actually send, must be less than or equal to the
amount reserved in reserved_points
@return: Deferred which fires when the payment has been scheduled
"""
self.queued_payments[reserved_points.identifier] += amount
# make any unused points available
self.total_reserved_points -= reserved_points.amount - amount
reserved_points.identifier.update_stats('points_sent', amount)
d = defer.succeed(True)
return d
def _send_queued_points(self):
ds = []
for peer, points in self.queued_payments.items():
if peer in self.peer_pub_keys:
d = pointtraderclient.send_points(
self.private_key, self.peer_pub_keys[peer], points)
self.wallet_balance -= points
self.total_reserved_points -= points
ds.append(d)
del self.queued_payments[peer]
else:
log.warning("Don't have a payment address for peer %s. Can't send %s points.",
str(peer), str(points))
return defer.DeferredList(ds)
def get_balance(self):
"""Return the balance of this wallet"""
d = pointtraderclient.get_balance(self.private_key)
return d
def add_expected_payment(self, peer, amount):
"""Increase the number of points expected to be paid by a peer"""
self.expected_payments[peer].append((amount, time.time()))
self.new_payments_expected_time = time.time()
peer.update_stats('expected_points', amount)
def set_public_key_for_peer(self, peer, pub_key):
self.peer_pub_keys[peer] = pub_key
def _get_new_payments(self):
def add_new_transactions(transactions):
for transaction in transactions:
if transaction[1] == self.encoded_public_key:
t_hash = SHA512.new()
t_hash.update(transaction[0])
t_hash.update(transaction[1])
t_hash.update(str(transaction[2]))
t_hash.update(transaction[3])
if t_hash.hexdigest() not in self.known_transactions:
self.known_transactions.append(t_hash.hexdigest())
self._add_received_payment(transaction[0], transaction[2])
d = pointtraderclient.get_recent_transactions(self.private_key)
d.addCallback(add_new_transactions)
return d
def _add_received_payment(self, encoded_other_public_key, amount):
self.received_payments[encoded_other_public_key].append((amount, time.time()))
def _check_good_standing(self):
for peer, expected_payments in self.expected_payments.iteritems():
expected_cutoff = time.time() - 90
min_expected_balance = sum([a[0] for a in expected_payments if a[1] < expected_cutoff])
received_balance = 0
if self.peer_pub_keys[peer] in self.received_payments:
received_balance = sum([
a[0] for a in self.received_payments[self.peer_pub_keys[peer]]])
if min_expected_balance > received_balance:
log.warning(
"Account in bad standing: %s (pub_key: %s), expected amount = %s, "
"received_amount = %s",
peer, self.peer_pub_keys[peer], min_expected_balance, received_balance)
def _open_db(self):
def open_db():
self.db = unqlite.UnQLite(os.path.join(self.db_dir, "ptcwallet.db"))
return threads.deferToThread(open_db)
def _save_private_key(self, private_key):
def save_key():
self.db['private_key'] = private_key
return threads.deferToThread(save_key)
def _get_wallet_private_key(self):
def get_key():
if 'private_key' in self.db:
return self.db['private_key']
return None
return threads.deferToThread(get_key)
class PointTraderKeyExchanger(object):
implements([IRequestCreator])
def __init__(self, wallet):
self.wallet = wallet
self._protocols = []
######### IRequestCreator #########
def send_next_request(self, peer, protocol):
if not protocol in self._protocols:
r = ClientRequest({'public_key': self.wallet.encoded_public_key},
'public_key')
d = protocol.add_request(r)
d.addCallback(self._handle_exchange_response, peer, r, protocol)
d.addErrback(self._request_failed, peer)
self._protocols.append(protocol)
return defer.succeed(True)
else:
return defer.succeed(False)
######### internal calls #########
def _handle_exchange_response(self, response_dict, peer, request, protocol):
assert request.response_identifier in response_dict, \
"Expected %s in dict but did not get it" % request.response_identifier
assert protocol in self._protocols, "Responding protocol is not in our list of protocols"
peer_pub_key = response_dict[request.response_identifier]
self.wallet.set_public_key_for_peer(peer, peer_pub_key)
return True
def _request_failed(self, err, peer):
if not err.check(RequestCanceledError):
log.warning("A peer failed to send a valid public key response. Error: %s, peer: %s",
err.getErrorMessage(), str(peer))
return err
class PointTraderKeyQueryHandlerFactory(object):
implements(IQueryHandlerFactory)
def __init__(self, wallet):
self.wallet = wallet
######### IQueryHandlerFactory #########
def build_query_handler(self):
q_h = PointTraderKeyQueryHandler(self.wallet)
return q_h
def get_primary_query_identifier(self):
return 'public_key'
def get_description(self):
return ("Point Trader Address - an address for receiving payments on the "
"point trader testing network")
class PointTraderKeyQueryHandler(object):
implements(IQueryHandler)
def __init__(self, wallet):
self.wallet = wallet
self.query_identifiers = ['public_key']
self.public_key = None
self.peer = None
######### IQueryHandler #########
def register_with_request_handler(self, request_handler, peer):
self.peer = peer
request_handler.register_query_handler(self, self.query_identifiers)
def handle_queries(self, queries):
if self.query_identifiers[0] in queries:
new_encoded_pub_key = queries[self.query_identifiers[0]]
try:
RSA.importKey(new_encoded_pub_key)
except (ValueError, TypeError, IndexError):
log.warning("Client sent an invalid public key.")
return defer.fail(Failure(ValueError("Client sent an invalid public key")))
self.public_key = new_encoded_pub_key
self.wallet.set_public_key_for_peer(self.peer, self.public_key)
log.debug("Received the client's public key: %s", str(self.public_key))
fields = {'public_key': self.wallet.encoded_public_key}
return defer.succeed(fields)
if self.public_key is None:
log.warning("Expected a public key, but did not receive one")
return defer.fail(Failure(ValueError("Expected but did not receive a public key")))
else:
return defer.succeed({})
-127
View File
@@ -1,127 +0,0 @@
from lbrynet.core.Strategy import get_default_strategy, OnlyFreeStrategy
from lbrynet import conf
from decimal import Decimal
class BasePaymentRateManager(object):
def __init__(self, rate=None, info_rate=None):
self.min_blob_data_payment_rate = rate if rate is not None else conf.settings['data_rate']
self.min_blob_info_payment_rate = (
info_rate if info_rate is not None else conf.settings['min_info_rate'])
class PaymentRateManager(object):
def __init__(self, base, rate=None):
"""
@param base: a BasePaymentRateManager
@param rate: the min blob data payment rate
"""
self.base = base
self.min_blob_data_payment_rate = rate
self.points_paid = 0.0
def get_rate_blob_data(self, peer):
return self.get_effective_min_blob_data_payment_rate()
def accept_rate_blob_data(self, peer, payment_rate):
return payment_rate >= self.get_effective_min_blob_data_payment_rate()
def get_effective_min_blob_data_payment_rate(self):
if self.min_blob_data_payment_rate is None:
return self.base.min_blob_data_payment_rate
return self.min_blob_data_payment_rate
def record_points_paid(self, amount):
self.points_paid += amount
class NegotiatedPaymentRateManager(object):
def __init__(self, base, availability_tracker, generous=None):
"""
@param base: a BasePaymentRateManager
@param availability_tracker: a BlobAvailabilityTracker
@param rate: the min blob data payment rate
"""
self.base = base
self.min_blob_data_payment_rate = base.min_blob_data_payment_rate
self.points_paid = 0.0
self.blob_tracker = availability_tracker
self.generous = generous if generous is not None else conf.settings['is_generous_host']
self.strategy = get_default_strategy(self.blob_tracker,
base_price=self.base.min_blob_data_payment_rate,
is_generous=generous)
def get_rate_blob_data(self, peer, blobs):
response = self.strategy.make_offer(peer, blobs)
return response.rate
def accept_rate_blob_data(self, peer, blobs, offer):
offer = self.strategy.respond_to_offer(offer, peer, blobs)
self.strategy.update_accepted_offers(peer, offer)
return offer.is_accepted
def reply_to_offer(self, peer, blobs, offer):
reply = self.strategy.respond_to_offer(offer, peer, blobs)
self.strategy.update_accepted_offers(peer, reply)
return reply
def get_rate_for_peer(self, peer):
return self.strategy.accepted_offers.get(peer, False)
def record_points_paid(self, amount):
self.points_paid += amount
def record_offer_reply(self, peer, offer):
self.strategy.update_accepted_offers(peer, offer)
def price_limit_reached(self, peer):
if peer in self.strategy.pending_sent_offers:
offer = self.strategy.pending_sent_offers[peer]
return (offer.is_too_low and
round(Decimal.from_float(offer.rate), 5) >= round(self.strategy.max_rate, 5))
return False
class OnlyFreePaymentsManager(object):
def __init__(self, **kwargs):
"""
A payment rate manager that will only ever accept and offer a rate of 0.0,
Used for testing
"""
self.base = BasePaymentRateManager(0.0, 0.0)
self.points_paid = 0.0
self.generous = True
self.strategy = OnlyFreeStrategy()
def get_rate_blob_data(self, peer, blobs):
response = self.strategy.make_offer(peer, blobs)
return response.rate
def accept_rate_blob_data(self, peer, blobs, offer):
offer = self.strategy.respond_to_offer(offer, peer, blobs)
self.strategy.update_accepted_offers(peer, offer)
return offer.is_accepted
def reply_to_offer(self, peer, blobs, offer):
reply = self.strategy.respond_to_offer(offer, peer, blobs)
self.strategy.update_accepted_offers(peer, reply)
return reply
def get_rate_for_peer(self, peer):
return self.strategy.accepted_offers.get(peer, False)
def record_points_paid(self, amount):
self.points_paid += amount
def record_offer_reply(self, peer, offer):
self.strategy.update_accepted_offers(peer, offer)
def price_limit_reached(self, peer):
if peer in self.strategy.pending_sent_offers:
offer = self.strategy.pending_sent_offers[peer]
if offer.rate > 0.0:
return True
return False
-46
View File
@@ -1,46 +0,0 @@
import datetime
from collections import defaultdict
from lbrynet.core import utils
# Do not create this object except through PeerManager
class Peer(object):
def __init__(self, host, port):
self.host = host
self.port = port
# If a peer is reported down, we wait till this time till reattempting connection
self.attempt_connection_at = None
# Number of times this Peer has been reported to be down, resets to 0 when it is up
self.down_count = 0
# Number of succesful connections (with full protocol completion) to this peer
self.success_count = 0
self.score = 0
self.stats = defaultdict(float) # {string stat_type, float count}
def is_available(self):
if self.attempt_connection_at is None or utils.today() > self.attempt_connection_at:
return True
return False
def report_up(self):
self.down_count = 0
self.attempt_connection_at = None
def report_success(self):
self.success_count += 1
def report_down(self):
self.down_count += 1
timeout_time = datetime.timedelta(seconds=60 * self.down_count)
self.attempt_connection_at = utils.today() + timeout_time
def update_score(self, score_change):
self.score += score_change
def update_stats(self, stat_type, count):
self.stats[stat_type] += count
def __str__(self):
return '{}:{}'.format(self.host, self.port)
def __repr__(self):
return 'Peer({!r}, {!r})'.format(self.host, self.port)
-19
View File
@@ -1,19 +0,0 @@
from twisted.internet import defer
class DummyPeerFinder(object):
"""This class finds peers which have announced to the DHT that they have certain blobs"""
def __init__(self):
pass
def run_manage_loop(self):
pass
def stop(self):
pass
def find_peers_for_blob(self, blob_hash):
return defer.succeed([])
def get_most_popular_hashes(self, num_to_return):
return []
-14
View File
@@ -1,14 +0,0 @@
from lbrynet.core.Peer import Peer
class PeerManager(object):
def __init__(self):
self.peers = []
def get_peer(self, host, port):
for peer in self.peers:
if peer.host == host and peer.port == port:
return peer
peer = Peer(host, port)
self.peers.append(peer)
return peer
-62
View File
@@ -1,62 +0,0 @@
from zope.interface import implementer
from decimal import Decimal
from lbrynet.interfaces import IBlobPriceModel
from lbrynet import conf
def get_default_price_model(blob_tracker, base_price, **kwargs):
return MeanAvailabilityWeightedPrice(blob_tracker, base_price, **kwargs)
class ZeroPrice(object):
def __init__(self):
self.base_price = 0.0
def calculate_price(self, blob):
return 0.0
class MeanAvailabilityWeightedPrice(object):
"""Calculate mean-blob-availability and stream-position weighted price for a blob
Attributes:
base_price (float): base price
alpha (float): constant, > 0.0 and <= 1.0, used to more highly
value blobs at the beginning of a stream.
alpha defaults to 1.0, which has a null effect
blob_tracker (BlobAvailabilityTracker): blob availability tracker
"""
implementer(IBlobPriceModel)
def __init__(self, tracker, base_price=None, alpha=1.0):
self.blob_tracker = tracker
base_price = base_price if base_price is not None else conf.settings['data_rate']
self.base_price = Decimal(base_price)
self.alpha = Decimal(alpha)
def calculate_price(self, blob):
mean_availability = self.blob_tracker.last_mean_availability
availability = self.blob_tracker.availability.get(blob, [])
index = 0 # blob.index
availability_mult = self._get_availability_multiplier(mean_availability, availability)
price = self.base_price * availability_mult / self._frontload(index)
return round(price, 5)
def _get_availability_multiplier(self, mean_availability, availability):
return Decimal(max(1, mean_availability) / Decimal(max(1, len(availability))))
def _frontload(self, index):
"""Get front-load multiplier, used to weight prices of blobs in a
stream towards the front of the stream.
At index 0, returns 1.0
As index increases, return value approaches 2.0
@param index: blob position in stream
@return: front-load multiplier
"""
return Decimal(2.0) - (self.alpha ** index)
-156
View File
@@ -1,156 +0,0 @@
import logging
from zope.interface import implements
from lbrynet.interfaces import IRateLimiter
from twisted.internet import task
log = logging.getLogger(__name__)
class DummyRateLimiter(object):
def __init__(self):
self.dl_bytes_this_second = 0
self.ul_bytes_this_second = 0
self.total_dl_bytes = 0
self.total_ul_bytes = 0
self.target_dl = 0
self.target_ul = 0
self.tick_call = None
def start(self):
self.tick_call = task.LoopingCall(self.tick)
self.tick_call.start(1)
def tick(self):
self.dl_bytes_this_second = 0
self.ul_bytes_this_second = 0
def stop(self):
if self.tick_call is not None:
self.tick_call.stop()
self.tick_call = None
def set_dl_limit(self, limit):
pass
def set_ul_limit(self, limit):
pass
def report_dl_bytes(self, num_bytes):
self.dl_bytes_this_second += num_bytes
self.total_dl_bytes += num_bytes
def report_ul_bytes(self, num_bytes):
self.ul_bytes_this_second += num_bytes
self.total_ul_bytes += num_bytes
class RateLimiter(object):
"""This class ensures that upload and download rates don't exceed specified maximums"""
implements(IRateLimiter)
#called by main application
def __init__(self, max_dl_bytes=None, max_ul_bytes=None):
self.max_dl_bytes = max_dl_bytes
self.max_ul_bytes = max_ul_bytes
self.dl_bytes_this_interval = 0
self.ul_bytes_this_interval = 0
self.total_dl_bytes = 0
self.total_ul_bytes = 0
self.tick_call = None
self.tick_interval = 0.1
self.dl_throttled = False
self.ul_throttled = False
self.protocols = []
def start(self):
log.info("Starting rate limiter.")
self.tick_call = task.LoopingCall(self.tick)
self.tick_call.start(self.tick_interval)
def tick(self):
self.dl_bytes_this_interval = 0
self.ul_bytes_this_interval = 0
self.unthrottle_dl()
self.unthrottle_ul()
def stop(self):
log.info("Stopping rate limiter.")
if self.tick_call is not None:
self.tick_call.stop()
self.tick_call = None
def set_dl_limit(self, limit):
self.max_dl_bytes = limit
def set_ul_limit(self, limit):
self.max_ul_bytes = limit
#throttling
def check_dl(self):
need_throttle = (self.max_dl_bytes is not None and
self.dl_bytes_this_interval > self.max_dl_bytes * self.tick_interval)
if need_throttle:
from twisted.internet import reactor
reactor.callLater(0, self.throttle_dl)
def check_ul(self):
need_throttle = (self.max_ul_bytes is not None and
self.ul_bytes_this_interval > self.max_ul_bytes * self.tick_interval)
if need_throttle:
from twisted.internet import reactor
reactor.callLater(0, self.throttle_ul)
def throttle_dl(self):
if self.dl_throttled is False:
for protocol in self.protocols:
protocol.throttle_download()
self.dl_throttled = True
def throttle_ul(self):
if self.ul_throttled is False:
for protocol in self.protocols:
protocol.throttle_upload()
self.ul_throttled = True
def unthrottle_dl(self):
if self.dl_throttled is True:
for protocol in self.protocols:
protocol.unthrottle_download()
self.dl_throttled = False
def unthrottle_ul(self):
if self.ul_throttled is True:
for protocol in self.protocols:
protocol.unthrottle_upload()
self.ul_throttled = False
#called by protocols
def report_dl_bytes(self, num_bytes):
self.dl_bytes_this_interval += num_bytes
self.total_dl_bytes += num_bytes
self.check_dl()
def report_ul_bytes(self, num_bytes):
self.ul_bytes_this_interval += num_bytes
self.total_ul_bytes += num_bytes
self.check_ul()
def register_protocol(self, protocol):
if protocol not in self.protocols:
self.protocols.append(protocol)
if self.dl_throttled is True:
protocol.throttle_download()
if self.ul_throttled is True:
protocol.throttle_upload()
def unregister_protocol(self, protocol):
if protocol in self.protocols:
self.protocols.remove(protocol)
-359
View File
@@ -1,359 +0,0 @@
import logging
import miniupnpc
from lbrynet.core.BlobManager import DiskBlobManager
from lbrynet.dht import node
from lbrynet.database.storage import SQLiteStorage
from lbrynet.core.PeerManager import PeerManager
from lbrynet.core.RateLimiter import RateLimiter
from lbrynet.core.client.DHTPeerFinder import DHTPeerFinder
from lbrynet.core.HashAnnouncer import DummyHashAnnouncer
from lbrynet.core.server.DHTHashAnnouncer import DHTHashAnnouncer
from lbrynet.core.utils import generate_id
from lbrynet.core.PaymentRateManager import BasePaymentRateManager, NegotiatedPaymentRateManager
from lbrynet.core.BlobAvailability import BlobAvailabilityTracker
from twisted.internet import threads, defer
log = logging.getLogger(__name__)
class Session(object):
"""This class manages all important services common to any application that uses the network.
the hash announcer, which informs other peers that this peer is
associated with some hash. Usually, this means this peer has a
blob identified by the hash in question, but it can be used for
other purposes.
the peer finder, which finds peers that are associated with some
hash.
the blob manager, which keeps track of which blobs have been
downloaded and provides access to them,
the rate limiter, which attempts to ensure download and upload
rates stay below a set maximum
upnp, which opens holes in compatible firewalls so that remote
peers can connect to this peer.
"""
def __init__(self, blob_data_payment_rate, db_dir=None,
node_id=None, peer_manager=None, dht_node_port=None,
known_dht_nodes=None, peer_finder=None,
hash_announcer=None, blob_dir=None,
blob_manager=None, peer_port=None, use_upnp=True,
rate_limiter=None, wallet=None,
dht_node_class=node.Node, blob_tracker_class=None,
payment_rate_manager_class=None, is_generous=True, external_ip=None, storage=None):
"""@param blob_data_payment_rate: The default payment rate for blob data
@param db_dir: The directory in which levelDB files should be stored
@param node_id: The unique ID of this node
@param peer_manager: An object which keeps track of all known
peers. If None, a PeerManager will be created
@param dht_node_port: The port on which the dht node should
listen for incoming connections
@param known_dht_nodes: A list of nodes which the dht node
should use to bootstrap into the dht
@param peer_finder: An object which is used to look up peers
that are associated with some hash. If None, a
DHTPeerFinder will be used, which looks for peers in the
distributed hash table.
@param hash_announcer: An object which announces to other
peers that this peer is associated with some hash. If
None, and peer_port is not None, a DHTHashAnnouncer will
be used. If None and peer_port is None, a
DummyHashAnnouncer will be used, which will not actually
announce anything.
@param blob_dir: The directory in which blobs will be
stored. If None and blob_manager is None, blobs will be
stored in memory only.
@param blob_manager: An object which keeps track of downloaded
blobs and provides access to them. If None, and blob_dir
is not None, a DiskBlobManager will be used, with the
given blob_dir. If None and blob_dir is None, a
TempBlobManager will be used, which stores blobs in memory
only.
@param peer_port: The port on which other peers should connect
to this peer
@param use_upnp: Whether or not to try to open a hole in the
firewall so that outside peers can connect to this peer's
peer_port and dht_node_port
@param rate_limiter: An object which keeps track of the amount
of data transferred to and from this peer, and can limit
that rate if desired
@param wallet: An object which will be used to keep track of
expected payments and which will pay peers. If None, a
wallet which uses the Point Trader system will be used,
which is meant for testing only
"""
self.db_dir = db_dir
self.node_id = node_id
self.peer_manager = peer_manager
self.dht_node_port = dht_node_port
self.known_dht_nodes = known_dht_nodes
if self.known_dht_nodes is None:
self.known_dht_nodes = []
self.peer_finder = peer_finder
self.hash_announcer = hash_announcer
self.blob_dir = blob_dir
self.blob_manager = blob_manager
self.blob_tracker = None
self.blob_tracker_class = blob_tracker_class or BlobAvailabilityTracker
self.peer_port = peer_port
self.use_upnp = use_upnp
self.rate_limiter = rate_limiter
self.external_ip = external_ip
self.upnp_redirects = []
self.wallet = wallet
self.dht_node_class = dht_node_class
self.dht_node = None
self.base_payment_rate_manager = BasePaymentRateManager(blob_data_payment_rate)
self.payment_rate_manager = None
self.payment_rate_manager_class = payment_rate_manager_class or NegotiatedPaymentRateManager
self.is_generous = is_generous
self.storage = storage or SQLiteStorage(self.db_dir)
def setup(self):
"""Create the blob directory and database if necessary, start all desired services"""
log.debug("Starting session.")
if self.node_id is None:
self.node_id = generate_id()
if self.wallet is None:
from lbrynet.core.PTCWallet import PTCWallet
self.wallet = PTCWallet(self.db_dir)
if self.peer_manager is None:
self.peer_manager = PeerManager()
if self.use_upnp is True:
d = self._try_upnp()
else:
d = defer.succeed(True)
if self.peer_finder is None:
d.addCallback(lambda _: self._setup_dht())
else:
if self.hash_announcer is None and self.peer_port is not None:
log.warning("The server has no way to advertise its available blobs.")
self.hash_announcer = DummyHashAnnouncer()
d.addCallback(lambda _: self._setup_other_components())
return d
def shut_down(self):
"""Stop all services"""
log.info('Stopping session.')
ds = []
if self.blob_tracker is not None:
ds.append(defer.maybeDeferred(self.blob_tracker.stop))
if self.dht_node is not None:
ds.append(defer.maybeDeferred(self.dht_node.stop))
if self.rate_limiter is not None:
ds.append(defer.maybeDeferred(self.rate_limiter.stop))
if self.peer_finder is not None:
ds.append(defer.maybeDeferred(self.peer_finder.stop))
if self.hash_announcer is not None:
ds.append(defer.maybeDeferred(self.hash_announcer.stop))
if self.wallet is not None:
ds.append(defer.maybeDeferred(self.wallet.stop))
if self.blob_manager is not None:
ds.append(defer.maybeDeferred(self.blob_manager.stop))
if self.use_upnp is True:
ds.append(defer.maybeDeferred(self._unset_upnp))
return defer.DeferredList(ds)
def _try_upnp(self):
log.debug("In _try_upnp")
def get_free_port(upnp, port, protocol):
# returns an existing mapping if it exists
mapping = upnp.getspecificportmapping(port, protocol)
if not mapping:
return port
if upnp.lanaddr == mapping[0]:
return mapping
return get_free_port(upnp, port + 1, protocol)
def get_port_mapping(upnp, internal_port, protocol, description):
# try to map to the requested port, if there is already a mapping use the next external
# port available
if protocol not in ['UDP', 'TCP']:
raise Exception("invalid protocol")
external_port = get_free_port(upnp, internal_port, protocol)
if isinstance(external_port, tuple):
log.info("Found existing UPnP redirect %s:%i (%s) to %s:%i, using it",
self.external_ip, external_port[1], protocol, upnp.lanaddr, internal_port)
return external_port[1], protocol
upnp.addportmapping(external_port, protocol, upnp.lanaddr, internal_port,
description, '')
log.info("Set UPnP redirect %s:%i (%s) to %s:%i", self.external_ip, external_port,
protocol, upnp.lanaddr, internal_port)
return external_port, protocol
def threaded_try_upnp():
if self.use_upnp is False:
log.debug("Not using upnp")
return False
u = miniupnpc.UPnP()
num_devices_found = u.discover()
if num_devices_found > 0:
u.selectigd()
external_ip = u.externalipaddress()
if external_ip != '0.0.0.0' and not self.external_ip:
# best not to rely on this external ip, the router can be behind layers of NATs
self.external_ip = external_ip
if self.peer_port:
self.upnp_redirects.append(
get_port_mapping(u, self.peer_port, 'TCP', 'LBRY peer port')
)
if self.dht_node_port:
self.upnp_redirects.append(
get_port_mapping(u, self.dht_node_port, 'UDP', 'LBRY DHT port')
)
return True
return False
def upnp_failed(err):
log.warning("UPnP failed. Reason: %s", err.getErrorMessage())
return False
d = threads.deferToThread(threaded_try_upnp)
d.addErrback(upnp_failed)
return d
# the callback, if any, will be invoked once the joining procedure
# has terminated
def join_dht(self, cb=None):
from twisted.internet import reactor
def join_resolved_addresses(result):
addresses = []
for success, value in result:
if success is True:
addresses.append(value)
return addresses
@defer.inlineCallbacks
def join_network(knownNodes):
log.debug("join DHT using known nodes: " + str(knownNodes))
result = yield self.dht_node.joinNetwork(knownNodes)
defer.returnValue(result)
ds = []
for host, port in self.known_dht_nodes:
d = reactor.resolve(host)
d.addCallback(lambda h: (h, port)) # match host to port
ds.append(d)
dl = defer.DeferredList(ds)
dl.addCallback(join_resolved_addresses)
dl.addCallback(join_network)
if cb:
dl.addCallback(cb)
return dl
def _setup_dht(self):
log.info("Starting DHT")
def start_dht(join_network_result):
self.hash_announcer.run_manage_loop()
return True
self.dht_node = self.dht_node_class(
udpPort=self.dht_node_port,
node_id=self.node_id,
externalIP=self.external_ip,
peerPort=self.peer_port
)
self.peer_finder = DHTPeerFinder(self.dht_node, self.peer_manager)
if self.hash_announcer is None:
self.hash_announcer = DHTHashAnnouncer(self.dht_node, self.peer_port)
self.dht_node.startNetwork()
# pass start_dht() as callback to start the remaining components after joining the DHT
return self.join_dht(start_dht)
def _setup_other_components(self):
log.debug("Setting up the rest of the components")
if self.rate_limiter is None:
self.rate_limiter = RateLimiter()
if self.blob_manager is None:
if self.blob_dir is None:
raise Exception(
"TempBlobManager is no longer supported, specify BlobManager or db_dir")
else:
self.blob_manager = DiskBlobManager(
self.hash_announcer, self.blob_dir, self.storage
)
if self.blob_tracker is None:
self.blob_tracker = self.blob_tracker_class(
self.blob_manager, self.peer_finder, self.dht_node
)
if self.payment_rate_manager is None:
self.payment_rate_manager = self.payment_rate_manager_class(
self.base_payment_rate_manager, self.blob_tracker, self.is_generous
)
self.rate_limiter.start()
d = self.storage.setup()
d.addCallback(lambda _: self.blob_manager.setup())
d.addCallback(lambda _: self.wallet.start())
d.addCallback(lambda _: self.blob_tracker.start())
return d
def _unset_upnp(self):
log.info("Unsetting upnp for session")
def threaded_unset_upnp():
u = miniupnpc.UPnP()
num_devices_found = u.discover()
if num_devices_found > 0:
u.selectigd()
for port, protocol in self.upnp_redirects:
if u.getspecificportmapping(port, protocol) is None:
log.warning(
"UPnP redirect for %s %d was removed by something else.",
protocol, port)
else:
u.deleteportmapping(port, protocol)
log.info("Removed UPnP redirect for %s %d.", protocol, port)
self.upnp_redirects = []
d = threads.deferToThread(threaded_unset_upnp)
d.addErrback(lambda err: str(err))
return d
Loaded 100 of 404 files, more files were not shown because too many files have changed in this diff. Show more