Compare commits

...
3180 Commits
Author SHA1 Message Date
Alex Grintsvayg 4e9977ac23 update staked_amount to include supports for claims in the channel's stake. fixes #3072
ci / lint (push) Waiting to run
ci / tests / unit (macos-latest) (push) Waiting to run
ci / tests / unit (ubuntu-latest) (push) Waiting to run
ci / tests / unit (windows-latest) (push) Waiting to run
ci / tests / integration (postgres, blockchain) (push) Waiting to run
ci / tests / integration (sqlite, blockchain) (push) Waiting to run
2020-12-18 14:57:22 -05:00
Victor Shyba 784343878b fix windows complaints on permissions for wallet file test 2020-12-15 14:40:56 -03:00
Victor Shyba 350eec35ee account for the number of variables on each iteration 2020-12-15 14:18:10 -03:00
Victor Shyba 76d0e31544 fixes for old keys 2020-12-11 19:31:39 -03:00
Victor Shyba cd3f189cf6 ClaimMeta attr typo 2020-12-08 20:50:37 -03:00
Victor Shyba 240f0a9a05 fix initial sync for sync without claims 2020-12-08 20:50:11 -03:00
Victor Shyba 94d76cb13d fix bug on index application 2020-12-08 18:34:40 -03:00
Lex Berezhny 4592e21424 minor refactoring 2020-12-07 07:40:16 -05:00
Victor Shyba 6df67f225f decrease usage of expensive call on tests 2020-12-07 07:40:16 -05:00
Victor Shyba c484a8abf5 fixes from review 2020-12-07 07:40:16 -05:00
Victor Shyba f4a8be6c19 use coalesce to high value instead of nulls last to be ANSI compatible 2020-12-07 07:40:16 -05:00
Victor Shyba 8968893216 reduce diff 2020-12-07 07:40:16 -05:00
Victor Shyba 09aa3039e9 fix compat issues between postgres/sqlite 2020-12-07 07:40:16 -05:00
Victor Shyba c1c9c2883a fix some of the tests 2020-12-07 07:40:16 -05:00
Victor Shyba 9c269c90bb accept FILTERING_CHANNEL_IDS and BLOCKING_CHANNEL_IDS env vars as the old SDK 2020-12-07 07:40:16 -05:00
Victor Shyba e4449d2ec7 refactor level to get_xyz_... 2020-12-07 07:40:16 -05:00
Victor Shyba 4e0728572d pylint 2020-12-07 07:40:16 -05:00
Victor Shyba d3ec200e46 support blocking content 2020-12-07 07:40:16 -05:00
Victor Shyba 90e7821283 filter whole channels 2020-12-07 07:40:16 -05:00
Victor Shyba a5eccb9a92 filter reposts 2020-12-07 07:40:16 -05:00
Victor Shyba f83289f876 add owner column, apply censor 2020-12-07 07:40:16 -05:00
Victor Shyba 6bb8a69efc load hashes into claim filter table 2020-12-07 07:40:16 -05:00
Victor Shyba 76b3bfe975 add simple trending 2020-12-07 07:40:16 -05:00
Victor Shyba f3710fa153 check all kwargs are consumed on claim search 2020-12-07 07:40:16 -05:00
Victor Shyba 2b62330980 fix channel argument 2020-12-07 07:40:16 -05:00
Victor Shyba 036e16d1fc decrease default to 999 until it updates 2020-12-01 16:20:05 -05:00
Victor Shyba ea691e78b8 batch under sqlite variable limit 2020-12-01 16:20:05 -05:00
Jack Robison 435a80d4c3 Update README.md 2020-12-01 12:34:04 -05:00
Lex Berezhny ec89baa831 lint fixes and revert sum_support overwrite during merging 2020-11-17 15:20:25 -05:00
Lex Berezhny 85cf19bb2d test_order_by fixed 2020-11-17 14:53:48 -05:00
Lex Berezhny 1971539369 test fixes 2020-11-17 14:53:48 -05:00
Lex Berezhny 5d4e61d089 reposts 2020-11-17 14:53:48 -05:00
Lex Berezhny 2ee65752b2 lint 2020-11-17 14:53:48 -05:00
Lex Berezhny 1d31a96c9b test_claim_commands.py 2020-11-17 14:53:48 -05:00
Lex Berezhny e6a9417988 transaction list 2020-11-17 14:53:48 -05:00
Lex Berezhny 1e4613fd8a more commands 2020-11-17 14:53:48 -05:00
Lex Berezhny d459d6a26b wallet commands progress 2020-11-17 14:53:48 -05:00
Lex Berezhny d8f5b11158 coalesce null balance 2020-11-17 14:53:48 -05:00
Lex Berezhny 6764d52634 postgres optimized balance query 2020-11-17 14:53:48 -05:00
Lex Berezhny 4d8cc494e1 add txo_unspent_by_address index 2020-11-17 14:53:48 -05:00
Lex Berezhny 22a04fab24 balance 2020-11-17 14:53:48 -05:00
Lex Berezhny 9f9fdd2d1a + include_total 2020-11-17 14:53:48 -05:00
Lex Berezhny 2ec91dbca7 send dict instead of Account and PubKey objects when adding keys 2020-11-17 14:53:48 -05:00
Lex Berezhny 1a4a2db1b1 command fixes 2020-11-17 14:53:48 -05:00
Lex Berezhny cc2837c021 account_add fix 2020-11-17 14:53:48 -05:00
Lex Berezhny 5cdfbcc88e full node wallet support 2020-11-17 14:53:48 -05:00
Lex Berezhny 6ed2fa20ec working db based wallet and wallet sync progress 2020-11-17 14:53:48 -05:00
Lex Berezhny 4356d23cc1 minor test fix 2020-11-17 14:53:48 -05:00
Lex Berezhny 2f4b5d2ffe lint 2020-11-17 14:53:48 -05:00
Lex Berezhny 0a06810f36 working mempool 2020-11-17 14:53:48 -05:00
Lex Berezhny 9ce8910b42 updated unit test imports 2020-11-17 14:53:48 -05:00
Lex Berezhny fa85558d71 wip 2020-11-17 14:53:48 -05:00
Alex Grintsvayg 999e4209fa add missing flag in usage 2020-11-11 16:59:53 -05:00
Alex Grintsvayg 7d82a4573f add indexes to optimize sum_supports query 2020-11-11 16:56:32 -05:00
Victor Shyba c1aa9b8150 ignore blocks marked as BLOCK_FAILED_VALID and BLOCK_FAILED_CHILD 2020-10-26 16:33:26 -04:00
Alex Grintsvayg daed032bb7 return total instead of percent so frontend can display percentage precisely 2020-10-21 10:34:18 -04:00
Alex Grintsvayg abebc0d878 add more tests, flag to exclude own supports 2020-10-21 10:34:18 -04:00
Alex Grintsvayg 4bb0344e05 add sum_supports api 2020-10-16 10:44:47 -04:00
Lex Berezhny 0215ce6b79 renamed expander functions to have different name from actual arguments by prepending extract_ 2020-10-05 17:24:02 -04:00
Lex Berezhny b9cf130344 fix for takeover height calculation 2020-09-25 15:02:31 -04:00
Lex Berezhny 85b8877933 Merge pull request #3039 from lbryio/fwss_fix_resolve
Include enough information in resolve protobuf so its possible to verify signatures
2020-09-24 20:57:56 -04:00
Lex Berezhny bba9aec4f2 added respoted_count calculation 2020-09-24 15:19:30 -04:00
Lex Berezhny 2853bb437d make TXs pickleable 2020-09-24 12:27:06 -04:00
Victor Shyba 2d7929f113 fixme->TODO 2020-09-23 14:39:24 -03:00
Victor Shyba 24386c74f4 fix test_nulls 2020-09-23 08:38:33 -04:00
Victor Shyba 0a2c161ace make short_url and canonical_url match old SDK 2020-09-23 08:38:33 -04:00
Victor Shyba 18b25f5146 add and test takeover info on resolve 2020-09-23 08:38:33 -04:00
Victor Shyba 1a5292aaf4 add expiration_height 2020-09-23 08:38:33 -04:00
Victor Shyba 39ba2a8b7f log errors for on_rpc 2020-09-23 08:38:33 -04:00
Victor Shyba 25b63c988f further fixes to effective amount 2020-09-23 08:38:33 -04:00
Victor Shyba fdac6416a1 tests: find_claim_txo for reliability in some places 2020-09-23 08:38:33 -04:00
Victor Shyba 4783e6da1f fix effective_amount 2020-09-23 08:38:33 -04:00
Victor Shyba 9c1d593e54 fix set_reference 2020-09-23 08:38:33 -04:00
Victor Shyba fa34ff88bc refactor db, make resolve censor right 2020-09-23 08:38:33 -04:00
Victor Shyba 6690e63ea4 remove commented code and unused import 2020-09-23 08:38:33 -04:00
Victor Shyba c7eb60619f separate test for not found message 2020-09-23 08:38:33 -04:00
Victor Shyba cb98fbc619 fix not found raising, add a test 2020-09-23 08:38:33 -04:00
Victor Shyba 4fcfa0b193 add channel information on resolve protobuf and test for its presence 2020-09-23 08:38:33 -04:00
Lex Berezhny 8cde120928 start --full-node -> start node 2020-09-23 08:37:45 -04:00
Lex Berezhny 052ef73226 pylint 2020-09-21 22:17:45 -04:00
Lex Berezhny 082a91dc15 use LEFT JOIN instead of IN for faster get_block_range_without_filters() 2020-09-21 22:04:50 -04:00
Lex Berezhny d10a88c79b script hash addresses 2020-09-21 21:46:41 -04:00
Lex Berezhny 62a3022a3d support for old public key addresses 2020-09-21 15:27:47 -04:00
Lex Berezhny 4e6bdb64a8 Merge pull request #3043 from lbryio/bip-158
Sync block filters
2020-09-21 12:02:50 -04:00
Lex Berezhny 0b34c4ea86 got tests to pass on sqlite and postgres and added filter groups at various block height scales 2020-09-21 10:26:19 -04:00
Jack Robison 5b9e40c0c0 sync block filters 2020-09-17 11:25:50 -04:00
Jack Robison 30dff02674 logging formatter 2020-09-17 11:25:28 -04:00
Lex Berezhny 20c7fe1e82 Merge pull request #3048 from lbryio/asyncio_loop_fix
asyncio loop fix
2020-09-16 20:16:13 -04:00
Lex Berezhny 23f186e6a1 pylint and other fixes 2020-09-16 20:02:34 -04:00
Lex Berezhny 14436b3955 instantiate objects inside async loop 2020-09-16 19:50:51 -04:00
Lex Berezhny a96ceba6f5 Daemon.from_config 2020-09-16 16:40:05 -04:00
Lex Berezhny f26aadbd44 move daemon construction from cli into daemon function 2020-09-16 16:34:55 -04:00
Lex Berezhny 144eb248e3 Merge pull request #3040 from lbryio/fwss_wait_lbrycrd
Synchronizer: wait for lbrycrd to start if its not ready, instead of raising
2020-09-14 13:21:10 -04:00
Victor Shyba a554c8838c use previous exception insted of declaring a new one 2020-09-14 14:08:59 -03:00
Victor Shyba 2a0089a4dd test starting and waiting normally as before too 2020-09-14 14:08:59 -03:00
Victor Shyba d6bcbd631f improve session close error message 2020-09-14 14:08:59 -03:00
Victor Shyba 39a4c4e590 exit if zmq misconfigured. tell whats wrong 2020-09-14 14:08:59 -03:00
Victor Shyba cb60cd99f4 use create_task instead 2020-09-14 14:08:59 -03:00
Victor Shyba a5c117b542 wait for lbrycrd on start 2020-09-14 14:08:59 -03:00
Victor Shyba 47a8c005d9 change zmq port for tests so it doesnt conflict with running lbrycrd 2020-09-14 14:08:59 -03:00
Lex Berezhny 6e50e83e28 Merge pull request #3044 from lbryio/fwss_pin_tqdm
pin tqdm and fix ovrride
2020-09-14 12:59:25 -04:00
Victor Shyba 50c8fc269b fix tqdm override and pin version 2020-09-14 13:46:31 -03:00
Alex Grintsvayg 336a0f6ae1 minor 2020-09-11 14:08:06 -04:00
Lex Berezhny e1c33dccab Merge pull request #3036 from lbryio/fwss_fix_parser
fix doc generator for docopt edge cases
2020-09-04 15:48:57 -04:00
Lex Berezhny 2f56f7f0e0 fix test 2020-09-04 15:40:59 -04:00
Lex Berezhny 28413742cc reduce diff 2 2020-09-04 15:38:31 -04:00
Lex Berezhny 6484894b36 reduce diff 2020-09-04 15:37:45 -04:00
Lex Berezhny c5fd9643f1 tx_kwargs expander should have wallet_id 2020-09-04 15:36:18 -04:00
Victor Shyba aa6d78b515 fix definitions, bump metadata 2020-09-04 15:09:28 -03:00
Lex Berezhny 33e266a0f4 unit test to make sure all generated help can be parsed by docopt 2020-09-04 10:57:11 -04:00
Lex Berezhny b4ee07162d made doc parsing error more helpful 2020-09-04 10:17:45 -04:00
Victor Shyba 0f63103db5 fix wallet_id parameter 2020-09-04 01:36:49 -03:00
Victor Shyba c3884352db improve error messages 2020-09-04 01:29:25 -03:00
Victor Shyba 8c525b6dfc fix tests 2020-09-04 01:25:56 -03:00
Victor Shyba d89b074615 bump metadata.py 2020-09-04 01:20:39 -03:00
Victor Shyba f775b0ed55 fix all comments with line breaks starting with -- 2020-09-04 01:20:27 -03:00
Victor Shyba 60c333c6e2 stream filter already have signed filter 2020-09-04 00:58:08 -03:00
Victor Shyba 42b7f8ff71 raise when -- ends up in a continuation line instead of auto fixing 2020-09-04 00:45:02 -03:00
Victor Shyba c981c767b9 raise instead of implicit deduplication of arguments 2020-09-04 00:44:55 -03:00
Victor Shyba 10ad4ed8d1 update metadata.py 2020-09-04 00:22:14 -03:00
Victor Shyba e436ae7edd test for wrong linebreaks and improve fix 2020-09-04 00:22:14 -03:00
Victor Shyba c498619cca test using a new expander explicitly 2020-09-04 00:22:14 -03:00
Victor Shyba 382facf264 test for duplicated kwargs 2020-09-04 00:22:14 -03:00
Victor Shyba 7304d24dfd dont break lines on -- to avoid docopt parsing issues 2020-09-04 00:22:14 -03:00
Victor Shyba b330c35282 remove duplicated kwargs from parser.py 2020-09-04 00:22:14 -03:00
Lex Berezhny 6dfa78afa3 Merge pull request #3034 from lbryio/wip_fwss
misc fixes
2020-09-03 23:21:15 -04:00
Victor Shyba 1c79daaafc linter: ignore raise-missing-from 2020-09-04 00:15:54 -03:00
Victor Shyba e1b55f017b remove traceback usage, add test 2020-09-03 15:21:53 -03:00
Victor Shyba 6bbfb45de7 return errors when websocket req fails 2020-09-02 20:38:30 -03:00
Victor Shyba 9b15799c72 makefile default to match the number of CPUs and not arbitrary 28 2020-09-02 18:50:22 -03:00
Victor Shyba bc678f2146 session close is async, fix annoying error message 2020-09-02 18:49:41 -03:00
Victor Shyba 7c5211d420 fix lbrycrdd url 2020-09-02 14:34:00 -03:00
Victor Shyba 70e50780c3 dropdb ignores already dropped db 2020-09-02 14:33:36 -03:00
Lex Berezhny 7ebb9d06df discover and run all unittests 2020-08-20 14:27:28 -04:00
Lex Berezhny b99de6b872 skip tests needing to be updated to use new APIs 2020-08-20 14:24:08 -04:00
Lex Berezhny 889464e51d drop components 2020-08-20 14:22:46 -04:00
Lex Berezhny 35c3ff1e30 test cleaup and fixing 2020-08-20 13:31:58 -04:00
Lex Berezhny aa75b9bb25 moved wallet specific unit tests into wallet dir 2020-08-20 13:30:33 -04:00
Lex Berezhny f995ceae8b pylint 2020-08-20 10:46:28 -04:00
Lex Berezhny c1803434aa support clean exit 2020-08-20 10:44:29 -04:00
Lex Berezhny 4dfbdcc2d7 Update docker-compose.yml 2020-08-14 00:29:51 -04:00
Lex Berezhny 272940b6d6 removed docker/hooks 2020-08-14 03:47:53 +00:00
Lex Berezhny baf384d6e0 cleanup 2020-08-14 03:42:04 +00:00
Lex Berezhny a012c04974 docker-compose.yml 2020-08-14 03:38:33 +00:00
Lex Berezhny 1fa117a104 upgrade distro 2020-08-14 00:33:44 +00:00
Lex Berezhny 33a157959d index added 2020-08-13 12:18:39 -04:00
Lex Berezhny 747eace4ab lint 2020-08-13 12:08:35 -04:00
Lex Berezhny 8fa2d746e7 re-enable all github workflows 2020-08-13 11:57:10 -04:00
Lex Berezhny a77b1f9997 move to docker dir 2020-08-13 11:50:20 -04:00
Lex Berezhny c914f24089 chmod lbrynet 2020-08-13 11:35:24 -04:00
Lex Berezhny e41a71a64e no dockerignore 2020-08-13 11:25:21 -04:00
Lex Berezhny e59e238fd5 dockerfile bin error 2020-08-13 11:17:45 -04:00
Lex Berezhny 4c331d00e7 moved Dockerfile 2020-08-13 00:40:12 -04:00
Lex Berezhny f56229bcce Dockerfile 2020-08-13 00:29:44 -04:00
Lex Berezhny abf1247f61 runs-on github action 2020-08-13 00:20:45 -04:00
Lex Berezhny 4b230a97f9 docker 2020-08-13 00:17:43 -04:00
Lex Berezhny bf7ac1562f docker 2020-08-12 03:34:37 -04:00
Lex Berezhny 2000d75c7a start --full-node 2020-08-12 00:08:06 -04:00
Lex Berezhny 1259002b51 systemd script installation 2020-08-11 22:50:24 -04:00
Lex Berezhny 70d9f4cf79 convert claim_id to channel_hash 2020-08-10 18:53:00 -04:00
Lex Berezhny 23723f8041 added support_search to api 2020-08-10 18:37:21 -04:00
Lex Berezhny 51a0f7ddc8 add index for release_time 2020-08-10 17:48:26 -04:00
Lex Berezhny 1228700487 typo 2020-08-10 14:10:15 -04:00
Lex Berezhny 67c5c192f3 fix order_by and include_totals 2020-08-10 14:03:12 -04:00
Lex Berezhny 6b9cf5b48c wip simulate_sync_console 2020-08-07 20:45:52 -04:00
Lex Berezhny 0a91bd35c5 db.resolve takes list 2020-08-06 11:45:23 -04:00
Lex Berezhny 84639cfb2e protobuf claim_search 2020-08-04 13:49:59 -04:00
Lex Berezhny 0b245aab31 use base64 for protobuf response 2020-08-04 11:47:03 -04:00
Lex Berezhny ee3db31541 protobuf resolve response 2020-08-04 10:41:49 -04:00
Lex Berezhny 8a3b960a85 delete tag when rewinding 2020-08-03 22:36:14 -04:00
Lex Berezhny 2952609972 fixed bug when maintenance sync spanned multiple files 2020-08-03 16:53:40 -04:00
Lex Berezhny 9ab8a7dd81 magic numbers 2020-08-03 12:24:13 -04:00
Lex Berezhny a7555932a9 Merge pull request #3012 from lbryio/unclosed_warnings_fix
fix unclosed transport warnings on tests
2020-08-03 10:33:51 -04:00
Victor Shyba 9411b26fd3 fix unclosed transport warnings on tests 2020-08-03 03:30:29 -03:00
Lex Berezhny 85db7d3ce7 lbrycrd print text response 2020-07-31 14:38:45 -04:00
Lex Berezhny 87f1d5b0ae added lbrycrd-rpc-user/pass 2020-07-30 14:51:04 -04:00
Lex Berezhny 64b8caeb5c cleanup lbrycrd 2020-07-29 11:46:44 -04:00
Lex Berezhny 3315175d1c raise unauthorized error when cannot connect to lbrycrd 2020-07-29 11:32:43 -04:00
Lex Berezhny a802d1f686 clear tasks 2020-07-29 00:41:40 -04:00
Lex Berezhny 684e389283 basicConfig 2020-07-29 00:35:39 -04:00
Lex Berezhny 895719a13d basic logging 2020-07-29 00:14:40 -04:00
Lex Berezhny 8f2cce7f61 removed extraneous @staticmethod annotation 2020-07-27 12:33:40 -04:00
Lex Berezhny 9114a9794d fix channel integration test 2020-07-27 12:19:12 -04:00
Lex Berezhny a1f3254261 lint 2020-07-27 11:52:24 -04:00
Lex Berezhny dbc0da2817 event generator and better ZMQ handling 2020-07-27 10:58:57 -04:00
Lex Berezhny 412ace1c6f remove assert causing failures in console.py 2020-07-22 18:07:18 -04:00
Lex Berezhny 8100efb48c expose api as POST not GET 2020-07-20 23:48:27 -04:00
Lex Berezhny 8d164dfed3 skipIf fix 2020-07-20 23:38:21 -04:00
Lex Berezhny ffea76cdd5 skip event buffering tests on all OSes except linux 2020-07-20 23:36:47 -04:00
Lex Berezhny 72ddb0c195 use normalized instead of claim_name for claim search 2020-07-20 22:44:18 -04:00
Lex Berezhny c57080711e added 99 as special number for no processes 2020-07-20 10:46:59 -04:00
Lex Berezhny 96aea579ac temporary debug print statements 2020-07-20 09:57:52 -04:00
Lex Berezhny e2aae23575 commit db transaction after setting work_mem 2020-07-16 23:27:24 -04:00
Lex Berezhny 2ae700feb3 vaccum claim after updating channel stats 2020-07-16 10:53:27 -04:00
Lex Berezhny d1ac066c6d include claim_hash in txo_claim_chahges postgres index 2020-07-16 10:53:03 -04:00
Lex Berezhny b61424979d lint 2020-07-15 23:50:52 -04:00
Lex Berezhny ca10874006 add vacuuming for tx table 2020-07-14 17:11:06 -04:00
Lex Berezhny a4680878c4 fixed issues with database transaction isolation 2020-07-14 16:46:24 -04:00
Lex Berezhny 1c29ae7204 run vacuum after every sync 2020-07-14 13:26:46 -04:00
Andrey Beletsky 86069b10ca Bind API server to the configured host instead of localhost 2020-07-14 14:56:47 +07:00
Lex Berezhny 9c5e2a8c8d fix none exception 2020-07-14 00:11:30 -04:00
Lex Berezhny 622a3b77ef indent 2020-07-14 00:05:25 -04:00
Lex Berezhny 0dff82c31c save all claims, even bad ones, also fix cli 2020-07-13 23:58:49 -04:00
Lex Berezhny 8e683c9cd0 indexing fixes 2020-07-13 21:00:24 -04:00
Lex Berezhny 69c45d43d3 fixed multi block file test 2020-07-13 18:32:44 -04:00
Lex Berezhny fab7b5579c skip test unreliable on mac 2020-07-13 18:21:41 -04:00
Lex Berezhny 9e87394fca lower event stress test params since it fails too easily on CI, attempt 3 2020-07-13 16:57:34 -04:00
Lex Berezhny 8c6633de17 lower event stress test params since it fails too easily on CI, attempt 2 2020-07-13 16:53:02 -04:00
Lex Berezhny 3af71a2674 lower event stress test params since it fails too easily on CI 2020-07-13 16:45:35 -04:00
Lex Berezhny b792b134a2 require ubuntu 20.04 for integration tests 2020-07-13 16:41:08 -04:00
Lex Berezhny f50196d395 fix channel integration test 2020-07-13 15:50:00 -04:00
Lex Berezhny 248e04089b pylint 2020-07-13 15:45:21 -04:00
Lex Berezhny 8fd92cb649 more indexes 2020-07-13 14:29:38 -04:00
Lex Berezhny af4138ff51 minor progress refinement 2020-07-13 13:12:01 -04:00
Lex Berezhny 462daf4dc4 increase work_mem and add more indexes 2020-07-13 12:38:28 -04:00
Lex Berezhny e63151a370 merged channel and content syncs and no longer storing canonical_url in column 2020-07-13 09:30:32 -04:00
Lex Berezhny 09a2b2fa46 wider labels 2020-07-13 00:59:44 -04:00
Lex Berezhny a3d91329fe performance 2020-07-13 00:55:30 -04:00
Lex Berezhny 7bf96fd637 silence invalid name errors when creating short_url 2020-07-12 18:07:12 -04:00
Lex Berezhny 5157b2535b correctly update main bar 2020-07-12 17:57:41 -04:00
Lex Berezhny 0151ce8040 use correct last progress bar state value 2020-07-12 17:29:58 -04:00
Lex Berezhny 5328ed105e reduce TX_FLUSH_SIZE 2020-07-12 16:44:36 -04:00
Lex Berezhny 7f01b1cb84 flipped main and secondary progress bars 2020-07-12 16:43:44 -04:00
Lex Berezhny 862c51946a test robustness 2020-07-12 16:28:13 -04:00
Lex Berezhny 1790ee3018 bug fixes and performance improvements 2020-07-12 16:27:51 -04:00
Lex Berezhny 7a4e5dcb05 handle claims in abandoned channels 2020-07-12 12:02:58 -04:00
Lex Berezhny 24a88db595 console fix 2020-07-12 09:22:06 -04:00
Lex Berezhny 915233c96c fixes 2020-07-12 08:55:25 -04:00
Lex Berezhny aa9365f218 refactored sync 2020-07-11 18:18:33 -04:00
Lex Berezhny 15b8891fce make resolve command work from cli 2020-07-07 10:52:41 -04:00
Lex Berezhny f8a8a75ae9 regenerated interface metadata 2020-07-07 10:49:40 -04:00
Lex Berezhny d18ed6c19b wrap short url creation in try/except 2020-07-07 00:20:11 -04:00
Lex Berezhny 4aa44d3b5a set work_mem for postgres 2020-07-06 22:42:15 -04:00
Lex Berezhny 34a9dff141 fixed conflict with two cursors when one is reading and another is commiting 2020-07-06 09:18:22 -04:00
Lex Berezhny 7d9bf03574 refactored 2020-07-05 23:03:45 -04:00
Lex Berezhny 3fe1981657 faster inputs/outputs sync 2020-07-03 15:47:25 -04:00
Lex Berezhny 192c79c49c capture stderr before process pool starts 2020-07-01 18:32:20 -04:00
Lex Berezhny 5883c9bc6c hide error from protobuf 2020-06-30 23:13:38 -04:00
Lex Berezhny 4b50d1e329 vacuum before setting inputs 2020-06-30 19:37:00 -04:00
Lex Berezhny 39d8a20fd5 pgcopy COPY command 2020-06-30 17:32:51 -04:00
Lex Berezhny 9ccf00f56b add pgcopy to setup.py 2020-06-30 12:18:03 -04:00
Lex Berezhny 46662b55c7 clean up build workflow 2020-06-30 12:17:48 -04:00
Lex Berezhny b45a222f98 event subscription and publishing bug fixes 2020-06-29 18:10:26 -04:00
Lex Berezhny 434c1bc6b3 test_nulls 2020-06-27 23:23:57 -04:00
Lex Berezhny 2495df8859 null in claim description 2020-06-27 23:14:28 -04:00
Lex Berezhny 635aebfeeb fixed null claim name support for postgres 2020-06-27 23:00:12 -04:00
Lex Berezhny 81926a42f9 fix postgres db connection 2020-06-27 22:31:49 -04:00
Lex Berezhny f2ff4410dc re-enable postgres integration tests 2020-06-27 22:24:59 -04:00
Lex Berezhny 564018c937 better debug badrow 2020-06-27 11:46:49 -04:00
Lex Berezhny 4d1eafc0a4 debug badrow 2020-06-27 09:39:23 -04:00
Lex Berezhny 211f8b2e59 pause console elapsed timer when no blocks read/save 2020-06-27 09:22:52 -04:00
Lex Berezhny 9500be26fd no order by file 2020-06-26 23:37:10 -04:00
Lex Berezhny 017ef5b41a order by file 2020-06-26 23:35:11 -04:00
Lex Berezhny 4b19861a74 console improvements 2020-06-26 21:52:01 -04:00
Lex Berezhny 5a0a987f0c refactored and simplified the blockchain sync 2020-06-26 10:39:58 -04:00
Lex Berezhny 4cb4659489 removed Takeovers for now 2020-06-26 10:39:16 -04:00
Lex Berezhny e64b108404 console 2020-06-23 13:11:16 -04:00
Lex Berezhny 7870abaef4 script for testing console progress bars 2020-06-23 12:28:01 -04:00
Lex Berezhny 71e14c8e63 least 2020-06-22 20:30:05 -04:00
Lex Berezhny b3b6361429 fix test 2020-06-22 18:47:10 -04:00
Lex Berezhny 0d5441f3bf update 2020-06-22 11:44:27 -04:00
Lex Berezhny 9757c69189 pylint 2020-06-22 11:02:59 -04:00
Lex Berezhny d8fb31aedd addeed greatest sql function that supports postgres and sqlite 2020-06-22 10:52:17 -04:00
Lex Berezhny 54a0bf9290 fixups to make lbrynet work on cli with postgres 2020-06-21 23:21:43 -04:00
Lex Berezhny a3ef8d7411 pylint 2020-06-21 20:14:14 -04:00
Lex Berezhny 4810ff5f94 run more tests on github workflow 2020-06-21 19:55:13 -04:00
Lex Berezhny 2306edebf7 tests/integration/blockchain/test_blockchain.py 2020-06-21 19:54:59 -04:00
Lex Berezhny db5a33dc3f blockchain/test_database.py 2020-06-21 19:54:34 -04:00
Lex Berezhny 597bebb5be tests/unit/db 2020-06-21 19:53:18 -04:00
Lex Berezhny 73ff1d3b3a run more tests 2020-06-21 19:52:59 -04:00
Lex Berezhny 9198877098 include_is_spent no longer needed 2020-06-21 19:52:39 -04:00
Lex Berezhny 46da2584ca db 2020-06-21 19:51:09 -04:00
Lex Berezhny 53b7d0a58b blockchain 2020-06-21 19:50:53 -04:00
Lex Berezhny d1a243247d progress 2020-06-19 14:28:34 -04:00
Lex Berezhny 18dc5fbc9f run crypto and schema tests 2020-06-10 15:45:57 -04:00
Lex Berezhny 410212c17a update tests 2020-06-10 15:45:29 -04:00
Lex Berezhny a39f87b3c5 more resilient test 2020-06-10 15:25:45 -04:00
Lex Berezhny 147b9d5ad1 run another integration test 2020-06-09 23:52:08 -04:00
Lex Berezhny 1f210c0b0b skip event queue test until it can be more reliable 2020-06-09 23:42:21 -04:00
Lex Berezhny 86df4bdd11 run more unittests 2020-06-09 23:28:07 -04:00
Lex Berezhny 096f74d79b silence expected exception 2020-06-09 23:27:45 -04:00
Lex Berezhny ae8bc59c65 remove duplicate dewies test 2020-06-09 23:27:19 -04:00
Lex Berezhny 01dbbb4c3a skip blockchain sync unittest 2020-06-09 23:26:59 -04:00
Lex Berezhny 96d1926da4 upgrad pyyaml 2020-06-09 23:25:42 -04:00
Lex Berezhny 2c10e71774 coverage on windows 2020-06-09 20:59:32 -04:00
Lex Berezhny ea6be53071 run all steps 2020-06-09 20:55:16 -04:00
Lex Berezhny cb5250f630 integration coverage 2020-06-09 20:48:38 -04:00
Lex Berezhny 54e83daa59 back to pinned coverallsapp 2020-06-09 18:13:27 -04:00
Lex Berezhny df44d6ef56 back to bboe 2020-06-09 17:13:07 -04:00
Lex Berezhny aa4ef94e15 andremiras coveralls - spacing tests-unit 2020-06-09 17:00:26 -04:00
Lex Berezhny fad144cb96 andremiras coveralls - spacing 2020-06-09 16:59:34 -04:00
Lex Berezhny 1fe444bca2 andremiras coveralls 2020-06-09 16:58:22 -04:00
Lex Berezhny 55196ccb6b master github action 2020-06-09 16:46:59 -04:00
Lex Berezhny b3cb50aff0 back to bboe 2020-06-09 16:30:55 -04:00
Lex Berezhny ebf36f513c re-check default coveralls package 2020-06-09 14:03:26 -04:00
Lex Berezhny 47d207ff77 bboe coveralls 2020-06-09 13:56:01 -04:00
Lex Berezhny d99e4221f2 pin coveralls app github action 2020-06-09 13:43:26 -04:00
Lex Berezhny 56ff1342c4 workflow typo 2020-06-09 13:01:18 -04:00
Lex Berezhny a8c8614948 workflow indentation 2020-06-09 13:00:24 -04:00
Lex Berezhny 0e9184048c try coveralls again 2020-06-09 12:58:05 -04:00
Lex Berezhny c01dceebcd try codecov python lib 2020-06-06 21:00:38 -04:00
Lex Berezhny e3cc6ea224 moved codecov into tox 2020-06-06 20:54:34 -04:00
Lex Berezhny 44bbd9578d sans build 2020-06-06 20:19:09 -04:00
Lex Berezhny 7855f9c93f sans unit tests 2020-06-06 20:02:31 -04:00
Lex Berezhny 76e21f65df fix codecov test flags 2020-06-06 13:39:03 -04:00
Lex Berezhny 53c8876b5e fix codecov test flags 2020-06-06 13:27:59 -04:00
Lex Berezhny 2927875830 placeholder classes to pass pylint 2020-06-06 13:20:26 -04:00
Lex Berezhny b9d954a394 install coveragge 2020-06-06 13:10:42 -04:00
Lex Berezhny a0fb3424aa more codecov experiments 2020-06-06 13:01:53 -04:00
Lex Berezhny 269c0f714e trying without codecov github action 2020-06-06 12:52:54 -04:00
Lex Berezhny 99d2a3f42b debugging codecov 2020-06-06 12:44:54 -04:00
Lex Berezhny a1aa578bc0 codecov2 2020-06-06 12:23:32 -04:00
Lex Berezhny 82062b5601 trying codecov 2020-06-06 12:20:19 -04:00
Lex Berezhny db1f984558 moved fast_sync.py script 2020-06-05 19:21:39 -04:00
Lex Berezhny e66445b46e deleted lbry.extras 2020-06-05 19:18:00 -04:00
Lex Berezhny 82b69109bd pylint 2020-06-05 00:51:55 -04:00
Lex Berezhny ec4e36446c update lbrycrd download URL 2020-06-05 00:38:57 -04:00
Lex Berezhny 3ef83febc0 progress 2020-06-05 00:35:22 -04:00
Lex Berezhny ffecd02fbc deleted db/search 2020-06-05 00:31:26 -04:00
Lex Berezhny ebddb1f0f5 moved exchange rate manager 2020-06-04 18:48:44 -04:00
Lex Berezhny 9d0b9805b2 github actions install psycopg2 2020-06-04 18:17:37 -04:00
Lex Berezhny 8653839c16 no tests run on github actions, just build binaries for now 2020-06-04 17:52:10 -04:00
Lex Berezhny 29b1f93699 switch cli to use pre-generated interface 2020-06-04 17:51:01 -04:00
Lex Berezhny 97c285f22b removed redundant assert equal 2020-06-04 17:50:21 -04:00
Lex Berezhny c12e07de11 stricter check for old windows directory structure 2020-06-04 17:49:59 -04:00
Lex Berezhny 726bae97b0 generated lbry.service.metadata 2020-06-04 17:29:34 -04:00
Lex Berezhny 5efb36ffd2 windows & mac path tests 2020-06-04 17:29:34 -04:00
Lex Berezhny b4d6b14599 + tqdm 2020-06-04 17:29:34 -04:00
Lex Berezhny 9a40381f5a imports 2020-06-04 17:29:34 -04:00
Lex Berezhny 47f6d542c5 import fix 2020-06-04 17:29:34 -04:00
Lex Berezhny 8799caa0e4 github actions 2020-06-04 17:29:34 -04:00
Lex Berezhny a042377a7b run unittest on multiple platforms 2020-06-03 19:43:08 -04:00
Lex Berezhny 596ed08395 EventQueuePublisher uses a buffer to reduce number of tasks created 2020-05-22 18:40:21 -04:00
Lex Berezhny 2af29b892b new cli 2020-05-20 18:05:49 -04:00
Lex Berezhny 7ffb169376 Service.run -> Daemon.run 2020-05-20 18:05:13 -04:00
Lex Berezhny d11f4f9bed switched Client class from plain GET RPC to WebSockets 2020-05-20 18:04:18 -04:00
Lex Berezhny f2e844c476 ignore warnings when parsing old claims 2020-05-20 18:02:47 -04:00
Lex Berezhny 83f7eab0e7 tests for directory configurability 2020-05-20 18:02:05 -04:00
Lex Berezhny 4a9f9906a0 event streams can now be "closed" and thus you can listen for .last event 2020-05-20 18:01:34 -04:00
Lex Berezhny b7ff6569e4 added Database.temp_sqlite_regtest 2020-05-20 17:59:26 -04:00
Lex Berezhny 12915143b8 improved directory structure configurability 2020-05-20 17:57:42 -04:00
Lex Berezhny 06d93e667a event reporting from blockchain.sync 2020-05-20 17:54:38 -04:00
Lex Berezhny b341187b14 moved necessary code to get download directory to upstream appdirs and deleted winpaths.py 2020-05-19 17:05:03 -04:00
Lex Berezhny a996e65eff integration tests updated 2020-05-18 08:29:15 -04:00
Lex Berezhny 0886a7946e unit tests updated 2020-05-18 08:28:23 -04:00
Lex Berezhny be6ebf0047 refactored wallet and mnemonic 2020-05-18 08:26:36 -04:00
Lex Berezhny 7c4f943bcb updated to testcase 2020-05-18 08:24:44 -04:00
Lex Berezhny 5b5c45ea76 API is now typed and includes sharable argument lists 2020-05-18 08:24:15 -04:00
Lex Berezhny 6986211c1e lbry.blockchain fixes and cleanup 2020-05-18 08:22:23 -04:00
Lex Berezhny 8ac78990d8 updated setup.py and lbry/__init__.py now includes imports to solve cyclic import issues 2020-05-18 08:20:13 -04:00
Lex Berezhny 4f879bbbae new cli 2020-05-18 08:15:53 -04:00
Lex Berezhny e3f080a7ad extract singular tags, languages, locations from kwargs 2020-05-18 08:15:24 -04:00
Lex Berezhny d232eeaf81 plain execute function 2020-05-18 08:14:30 -04:00
Lex Berezhny 5d6388b366 added db_url to config 2020-05-18 08:13:29 -04:00
Lex Berezhny 955e44631d new API parser 2020-05-12 11:02:34 -04:00
Lex Berezhny 8dc5150dbe moved test fixture file 2020-05-12 10:35:34 -04:00
Lex Berezhny d488bfd9d4 refactored mnemonic.py 2020-05-09 18:33:23 -04:00
Lex Berezhny 391b95fd12 lbry.wallet.words minified removing line breaks and extra space and converted to tuples 2020-05-08 16:25:56 -04:00
Lex Berezhny 4b172e4180 updated testcase to use api instead of daemon 2020-05-08 10:34:57 -04:00
Lex Berezhny 29ef4425b0 fix import in managed_stream 2020-05-08 10:33:41 -04:00
Lex Berezhny 558b1aeadf made lbry.db.queries.add_channel_keys_to_txo_results async 2020-05-08 10:30:40 -04:00
Lex Berezhny 41ce3e4ad8 fix Output source offset calculation and made generat_channel_private_key async 2020-05-08 10:23:50 -04:00
Lex Berezhny 8c91777e5d refactored lbry.wallet 2020-05-06 10:53:31 -04:00
Lex Berezhny db89607e4e added create_default_wallet and create_default_account conf options 2020-05-06 10:50:29 -04:00
Lex Berezhny d476f08d13 convenience methods on ledger 2020-05-06 10:50:00 -04:00
Lex Berezhny 2a0c653c37 claim_search progress 2020-05-01 23:25:07 -04:00
Lex Berezhny 219c7cf37d removed lbry.wallet.server and lbry.wallet.orchstr8 2020-05-01 23:22:17 -04:00
Lex Berezhny 2f575a393f minor updates to stream_manager.py 2020-05-01 09:34:57 -04:00
Lex Berezhny 713c665588 unittests 2020-05-01 09:34:34 -04:00
Lex Berezhny 9554b66a37 wip lbry.service 2020-05-01 09:33:58 -04:00
Lex Berezhny 533f31cc89 wip lbry.wallet 2020-05-01 09:33:10 -04:00
Lex Berezhny fef09c1773 wip misc 2020-05-01 09:31:33 -04:00
Lex Berezhny 6a33d86bfe wip lbry.db 2020-05-01 09:29:44 -04:00
Lex Berezhny ccd32eae70 wip lbry.blockchain 2020-05-01 09:28:51 -04:00
Lex Berezhny c61c9726b0 lbry/blockchain/constants.py -> lbry/constants.py 2020-04-28 13:28:30 -04:00
Lex Berezhny fd5be69d55 lbry/wallet/server/db/{reader,fts} -> lbry/db 2020-04-28 12:26:17 -04:00
Lex Berezhny 1f72751a88 unit test __init__.py files 2020-04-25 08:58:18 -04:00
Lex Berezhny 362ab67186 more wallet unit test cleanup 2020-04-25 08:51:41 -04:00
Lex Berezhny ffe7fcf124 refactored wallet unit tests, moved most things to blockchain 2020-04-25 08:44:36 -04:00
Lex Berezhny 1f5dbc3eb8 renamed: wallet/stream.py -> event.py 2020-04-25 08:29:25 -04:00
Lex Berezhny 4dd85a169b cleaning up wallet directory, mostly moving things to blockchain 2020-04-25 08:25:29 -04:00
Lex Berezhny fe547f1b0e dropping custom wallet rpc implementation 2020-04-25 08:05:46 -04:00
Lex Berezhny ba154c799e increase timeout for wallet server payment service 2020-04-12 16:14:18 -04:00
Lex Berezhny e2ffd24d51 listen for wallet pay service payment earlier in test 2020-04-12 15:51:18 -04:00
Lex Berezhny 8545ab880b psycopg2 returns Decimal for SUM() 2020-04-12 15:30:25 -04:00
Lex Berezhny 4e85f34353 pylint 2020-04-12 12:05:03 -04:00
Lex Berezhny 777c6342f8 merged some old stashed code 2020-04-12 11:59:00 -04:00
Lex Berezhny 10c262a095 postgres 2020-04-12 11:06:05 -04:00
Lex Berezhny 74e3471bd9 setup.py 2020-04-12 09:26:59 -04:00
Lex Berezhny 4048cfb3e8 Update node.py 2020-04-11 21:20:02 -04:00
Lex Berezhny 8b7b284c0d Update setup.py 2020-04-11 21:07:07 -04:00
Lex Berezhny 2d4d51388b Update setup.py 2020-04-11 20:54:08 -04:00
Lex Berezhny 06b75d07dc sudo 2020-04-11 20:21:28 -04:00
Lex Berezhny 87a44fd41c syntax 2020-04-11 20:20:21 -04:00
Lex Berezhny 70693f4d1a apt-get update 2020-04-11 20:17:55 -04:00
Lex Berezhny 42224dadb6 pylint 2020-04-11 20:15:04 -04:00
Lex Berezhny 53fc94d688 pylint and sqlalchemy 1.4 2020-04-11 20:01:10 -04:00
Lex Berezhny 0b6d01fecc sqlalchemy 2020-04-11 17:31:20 -04:00
Lex Berezhny ae9d4af8c0 postgresql support added 2020-04-11 17:27:41 -04:00
Lex Berezhny 2309d6354c schema chaining 2020-04-08 19:03:01 -04:00
Lex Berezhny fd2f9846e9 parsing and inserting works 2020-04-08 19:03:01 -04:00
Lex Berezhny 66666e1167 script to generate blockchain test data 2020-04-08 19:03:01 -04:00
Lex Berezhny 4c0fbb84d6 reproduce issue with lbrycrd -reindex 2020-04-08 19:03:01 -04:00
Lex Berezhny fd3448ffb8 remove plyvel and add pyzmq 2020-04-08 19:03:01 -04:00
Lex Berezhny 6d93f97b51 Transaction._deserialize now optionally accepts stream and returns self 2020-04-08 19:03:01 -04:00
Lex Berezhny 2ee13ce39f BCDataStream can now also take file pointer as argument 2020-04-08 19:03:01 -04:00
Lex Berezhny bad4320ddf initial import of fast sync 2020-04-08 19:03:01 -04:00
Lex Berezhny a220736dea updated docs 2020-04-08 12:57:24 -04:00
Alex Grintsvayg 149d343201 drop a few unused conf vars 2020-04-07 16:11:23 -04:00
Lex Berezhny 496cc79ba8 v0.68.0 2020-04-06 12:55:21 -04:00
Jack Robison 3591768745 Merge pull request #2908 from lbryio/batched-resolve
Fix connection to wallet server breaking upon giant resolve requests
2020-04-06 12:52:51 -04:00
Jack Robison d615f6761a automatically batch large resolve requests 2020-04-06 12:03:33 -04:00
Jack Robison 5b29894048 add reset clients counter to prometheus 2020-04-06 12:03:33 -04:00
Lex Berezhny d17f0cfa40 Merge pull request #2909 from lbryio/none_for_unconfirmed_time
do not estimate timestamp for unconfirmed txs
2020-04-06 11:41:17 -04:00
Victor Shyba b2f70c7120 return none for unconfirmed time estimation 2020-04-06 06:03:35 -03:00
Jack Robison 238707bd93 Merge pull request #2897 from lbryio/reorg-claims
handle claim reorgs in the wallet server
2020-04-05 17:48:56 -04:00
Jack Robison 87cdf1e3a0 improve test_reorg_change_claim_height 2020-04-05 16:58:36 -04:00
Jack Robison 57fd47022e test_reorg_change_claim_height 2020-04-03 13:39:44 -04:00
Jack Robison 5eafd3bf6b feedback 2020-04-03 13:39:44 -04:00
Jack Robison 640b5b0ea9 delete_claims_above_height with thread lock 2020-04-03 13:39:44 -04:00
Jack Robison e4fb2f4680 test_reorg_dropping_claim 2020-04-03 13:39:44 -04:00
Jack Robison a4909f54e4 test reorg count metric 2020-04-03 13:39:44 -04:00
Jack Robison f7065c6f0c add reorg count metric to prometheus 2020-04-03 13:39:44 -04:00
Jack Robison 64f7f837e7 delete claims above reorg height from the database 2020-04-03 13:39:44 -04:00
Jack Robison 130d36acd9 Merge pull request #2905 from lbryio/testnet-headers
fix header checkpoints on testnet
2020-04-03 13:38:47 -04:00
Jack Robison e94c28cfa2 fix header checkpoints on testnet 2020-04-03 12:21:17 -04:00
Jack Robison a697dec30c Merge pull request #2902 from lbryio/bump_checkpoints
bump checkpoints
2020-04-03 12:20:38 -04:00
Victor Shyba bac09e9b9f bump checkpoints 2020-04-03 12:26:52 -03:00
Lex Berezhny b179f8f86a Merge pull request #2901 from lbryio/txo_is_spent_filter
replaced `--unspent` with `--is_not_spent` on several commands and added `--is_spent` to many claim/txo `*_list` commands, `support_list` got three new flags `--sent --received --staked`
2020-04-02 22:24:47 -04:00
Lex Berezhny 962d04ae17 fix txo_spend 2020-04-01 21:03:56 -04:00
Lex Berezhny f28e3bfe37 lint 2020-04-01 20:53:09 -04:00
Lex Berezhny 6474c86d32 cleaned up *_list commands 2020-04-01 20:44:34 -04:00
Lex Berezhny 769ea8cdfe added --is_spent filter to txo list/sum commands 2020-03-31 23:08:51 -04:00
Lex Berezhny 052e77dd5a v0.67.2 2020-03-31 17:47:19 -04:00
Lex Berezhny 1a4b0cb12a Merge pull request #2900 from lbryio/wallet_status_does_not_fail
`wallet_status` no longer fails if wallet component has not started
2020-03-31 17:45:51 -04:00
Lex Berezhny f9aa95c987 default to None for all values in wallet_status when wallet_manager not started yet 2020-03-31 17:22:13 -04:00
Lex Berezhny a5d06fb4a4 wallet_status no longer fails if wallet component has not started 2020-03-31 16:20:13 -04:00
Lex Berezhny d5e5d90bdc v0.67.1 2020-03-31 12:38:38 -04:00
Lex Berezhny e62678e4e6 Merge pull request #2898 from lbryio/duration_is_int
`ffmpeg` file analysis returns duration as integer now
2020-03-31 12:06:44 -04:00
Lex Berezhny 558ac24f7e fix test 2020-03-31 11:29:58 -04:00
Lex Berezhny 6f22f6a59f use ceil() on duration float() instead of int() directly 2020-03-31 10:57:37 -04:00
Lex Berezhny 5ec74f8abe ffmpeg file analysis returns duration as integer now 2020-03-31 10:28:04 -04:00
Lex Berezhny 16d7547e03 changed default txo_spend batch_size default to 500 from 1000 2020-03-30 22:48:05 -04:00
Lex Berezhny 767112dcda updated txo_spend docs 2020-03-30 21:45:58 -04:00
Lex Berezhny 81e23d1d8c v0.67.0 2020-03-30 19:43:45 -04:00
Lex Berezhny ace90a4354 Merge pull request #2890 from lbryio/faster-status
improve speed of `status`
2020-03-30 19:42:57 -04:00
Jack Robison 267e7096cc add test for rechecking ffmpeg installation 2020-03-30 19:11:04 -04:00
Jack Robison e7cded7511 check ffmpeg/ffmprobe paths in a thread 2020-03-30 19:11:04 -04:00
Jack Robison 33fbd715c0 don't block status on connectivity check 2020-03-30 19:11:04 -04:00
Jack Robison 25ba5b867c dont recheck ffmpeg installation in status 2020-03-30 19:11:04 -04:00
Lex Berezhny c8f431447c Merge pull request #2885 from lbryio/repair_tip_on_open
repair headers tip on open and let new headers come in during background fetching
2020-03-30 19:03:09 -04:00
Victor Shyba 1b83a1d09a test and fix verifying from middle 2020-03-30 19:42:27 -03:00
Victor Shyba d2fb7a7151 lock only when fetching, giving a chance for tip updates 2020-03-30 19:42:27 -03:00
Victor Shyba 7ad3447598 repair tip on open 2020-03-30 19:42:27 -03:00
Lex Berezhny 5e5bc8e705 Merge pull request #2887 from lbryio/txo_spend
added `txo_spend` command to support liquidating large number of txos (eg. tips) by batching them across several transactions
2020-03-30 18:41:05 -04:00
Lex Berezhny 886d1e8a19 added --include_full_tx option to txo_list 2020-03-30 18:15:13 -04:00
Lex Berezhny 6de7a035fa added preview/blocking back into doc string 2020-03-30 17:47:38 -04:00
Lex Berezhny 48d2497eb2 added txo_spend command to support liquidating large number of txos (eg. tips) 2020-03-30 17:47:38 -04:00
Lex Berezhny 6494754ab9 Merge pull request #2896 from lbryio/first_on_ready
fix for startup issues when connecting to wallet servers
2020-03-30 17:41:14 -04:00
Lex Berezhny a8153627c6 move on_read.first to earlier 2020-03-30 17:02:08 -04:00
Lex Berezhny ed38966edb add test to verify we listen to on_ready before it actually triggers 2020-03-30 15:26:04 -04:00
Lex Berezhny 85551d1e54 remove no-op test, segwit is always on now 2020-03-30 15:17:25 -04:00
Lex Berezhny ca31363180 listen for on_read.first before it is triggered 2020-03-30 14:53:52 -04:00
Lex Berezhny a8f11eb3c3 Merge pull request #2893 from lbryio/fix_float_timestamp
fix estimated timestamp to return integer instead of float
2020-03-30 10:58:32 -04:00
Lex Berezhny 151805121c increase wallet server payment test timeout 2020-03-29 20:00:23 -04:00
Lex Berezhny 0422d2a021 estimated timestamp should be integer 2020-03-29 19:39:37 -04:00
Lex Berezhny ffdf70257b Merge pull request #2888 from lbryio/m4a-content-type
add m4a content type
2020-03-27 17:54:45 -04:00
Thomas Zarebczan 6079d60aa1 add m4a content type 2020-03-26 23:08:52 -04:00
Jack Robison d46d012f8c Revert "delete live bittrex test"
This reverts commit f70343bb
2020-03-26 20:31:39 -04:00
Jack Robison baf926b360 v0.66.0 2020-03-26 17:59:02 -04:00
Jack Robison 82339869dc Revert "v0.66.0"
This reverts commit 4ab56ae3
2020-03-26 17:50:44 -04:00
Jack Robison f70343bb63 delete live bittrex test 2020-03-26 17:50:31 -04:00
Jack Robison 4ab56ae3d1 v0.66.0 2020-03-26 16:49:54 -04:00
Jack Robison 39b4031684 Merge pull request #2880 from lbryio/handle_mkv_files
Handle MKV files
2020-03-26 16:45:20 -04:00
Brannon King 71f8965393 re-use ffprobe info in stream_type
avoid duplicate args


handle review comment
2020-03-26 16:20:12 -04:00
Brannon King 66857e72a4 ensure only webm-type MKV files 2020-03-26 16:20:12 -04:00
Jack Robison 15eb5d47eb Merge pull request #2881 from lbryio/wallet_sync_status
add `wallet_syncing` to `status`, which is set to true when processing transaction updates
2020-03-26 15:28:20 -04:00
Jack Robison 86242139da update tests 2020-03-26 13:56:51 -04:00
Victor Shyba 460bdc4148 move wallet_syncing to wallet status is_syncing 2020-03-26 11:43:48 -04:00
Victor Shyba 1052126522 add wallet_syncing status 2020-03-26 11:43:48 -04:00
Jack Robison fdb42ac876 Merge pull request #2852 from lbryio/faster-transaction-sync
More efficient syncing of wallet transactions
2020-03-26 11:43:20 -04:00
Jack Robison 6486f986e8 bump min required server version 2020-03-26 11:41:49 -04:00
Jack Robison bc4075e2ed create_task instead of ensure_future 2020-03-26 11:41:49 -04:00
Jack Robison 11525c7d0d fix caching transaction_show 2020-03-26 11:41:49 -04:00
Jack Robison bb12ae9ce6 fix duplicate get_merkle requests 2020-03-26 11:41:49 -04:00
Jack Robison 66b4ad0c44 improve log 2020-03-26 11:41:49 -04:00
Jack Robison 4bff2d718e clean up test 2020-03-26 11:41:49 -04:00
Jack Robison af7a7b4dc6 fix maybe_migrate_certificates blocking the api while populating channel signing keys 2020-03-26 11:41:49 -04:00
Jack Robison ec20d9a2a8 faster wallet sync with get_transaction_and_merkle 2020-03-26 11:41:49 -04:00
Lex Berezhny b0aee3d335 Merge pull request #2884 from lbryio/txo_plot
added `txo_plot` command to allow plotting txo sums over time
2020-03-26 10:14:02 -04:00
Lex Berezhny 5e2ddbfd86 txo_plot returns lbc instead of dewies 2020-03-26 01:13:09 -04:00
Lex Berezhny f086ebbb8e lint 2020-03-26 00:42:11 -04:00
Lex Berezhny e5bf6a5bfc added txo_plot command to allow plotting txo sums over time 2020-03-26 00:37:13 -04:00
Lex Berezhny 76fa86d54b v0.65.0 2020-03-23 14:56:45 -04:00
Jack Robison 60d89506a5 Merge pull request #2877 from lbryio/on_ready_fixup
wait for tx sync to finish but not header sync
2020-03-23 13:46:47 -04:00
Victor Shyba 2893f1eb9e add other taskgroup for non-start-important tasks 2020-03-23 12:36:05 -03:00
Victor Shyba a08cbf412d on_ready.wait so we start after tx syncs 2020-03-23 12:26:07 -03:00
Lex Berezhny 0bd65356f9 increase timeout in usage payment test 2020-03-23 11:06:29 -04:00
Lex Berezhny 2e2b39455f Merge pull request #2875 from lbryio/1080p
default to 1080P for anything higher than 1080P
2020-03-23 10:44:18 -04:00
Lex Berezhny 1f9fbe34e4 Merge pull request #2863 from lbryio/async_headers
fetch headers on demand
2020-03-23 10:41:31 -04:00
Thomas Zarebczan 506d7ae50e default to 1080P for anything higher than 1080P
Previously would do 1440p
2020-03-23 10:20:13 -04:00
Victor Shyba 342cb00625 less concurrent repeated header checks 2020-03-23 01:19:34 -03:00
Victor Shyba 952fc01efd add script that generates checkpoints 2020-03-23 00:05:36 -03:00
Victor Shyba 3eebe301fe move checkpoints out of folder into file 2020-03-22 23:45:14 -03:00
Victor Shyba 19c0a81c42 fix bad usages of hash and some tests 2020-03-22 23:31:47 -03:00
Victor Shyba 9fc7f9904b fix tests, delete old code 2020-03-22 23:31:47 -03:00
Victor Shyba b04a516063 better locking, stop corrupting headers, fix some tests 2020-03-22 23:31:47 -03:00
Victor Shyba 241e946d91 first experimental version 2020-03-22 23:31:45 -03:00
Victor Shyba e45375dc26 more async parts 2020-03-22 23:30:11 -03:00
Victor Shyba ec8e243323 estimate timestamps instead of using block headers 2020-03-22 23:30:11 -03:00
Victor Shyba af0e9368d4 headers get now async 2020-03-22 23:30:11 -03:00
Lex Berezhny 285483b81a Merge pull request #2872 from lbryio/claim_list_include_received_tips
added `--include_received_tips` to `claim_list` and `txo_list` commands
2020-03-22 18:00:35 -04:00
Lex Berezhny 73a91d5569 added --include_received_tips to claim_list/txo_list 2020-03-22 17:22:15 -04:00
Lex Berezhny e24c78be32 Merge pull request #2871 from lbryio/wallet_txo_data_leak
fix to prevent transaction cache from leaking some information about outputs between unrelated wallets
2020-03-22 13:16:03 -04:00
Lex Berezhny b11184de68 fix purchase_create 2020-03-22 12:23:21 -04:00
Lex Berezhny 61d02fc5d7 fix to prevent transaction cache from leaking some information about outputs between unrelated wallets 2020-03-22 11:45:40 -04:00
Lex Berezhny f0217f6821 Merge pull request #2870 from lbryio/resolve_includes
added new flags to `resolve` (and a few to `claim_search`) commands: `--include_purchase_receipt`, `--include_is_my_output`, `--include_sent_supports`, `--include_sent_tips` and `--include_received_tips`
2020-03-22 02:08:28 -04:00
Lex Berezhny 09644914a6 added --include_is_my_output to claim_search 2020-03-22 01:51:09 -04:00
Lex Berezhny b5c24d6a48 fixing unit tests 2020-03-22 01:29:26 -04:00
Lex Berezhny bdd2ac2c25 renamed some flags 2020-03-22 01:13:26 -04:00
Lex Berezhny 76376f0d33 lint 2020-03-22 00:24:38 -04:00
Lex Berezhny 9749da46ae added flags to resolve: --include_purchase_receipt, --include_is_my_output, --include_my_supports, --include_my_tips 2020-03-21 23:44:57 -04:00
Lex Berezhny 1c05295e89 updated doc 2020-03-21 20:15:21 -04:00
Lex Berezhny 4d0f28215a added release-text-lines: to release script to support multi-line release notes 2020-03-21 20:09:39 -04:00
Lex Berezhny 0bba72bc5a Merge pull request #2862 from lbryio/txo_list_support_sent_outputs
`txo_list` adds many new ownership filters such as `--is_my_input`, `--is_my_output`, etc and some metadata filters such as `--channel_id`, `--reposted_claim_id`, etc and a new command `txo_sum` which takes the same arguments as `txo_list` and produces sum of outputs
2020-03-21 19:48:34 -04:00
Lex Berezhny 87089b8e83 fix support test 2020-03-21 18:48:06 -04:00
Lex Berezhny 6a58148a89 added support for --order_by=none 2020-03-21 18:16:25 -04:00
Lex Berezhny 15091052be added --no_totals to txo_list 2020-03-21 18:06:05 -04:00
Lex Berezhny 5cd7e9a9b8 increment scema version and force specific indexes to be used for get_txos() 2020-03-21 15:08:14 -04:00
Lex Berezhny 7cb530c334 added --channel_id and --order_by to txo_list 2020-03-20 23:19:26 -04:00
Lex Berezhny 5e0324cc91 added --reposted_claim_id to txo_list 2020-03-20 20:22:57 -04:00
Lex Berezhny 6293e227ea added txo_sum command 2020-03-20 19:08:47 -04:00
Lex Berezhny d6d83a5c76 integration test fix 2020-03-20 19:08:47 -04:00
Lex Berezhny 93fc883b90 fixing unit tests 2020-03-20 19:08:47 -04:00
Lex Berezhny dd21803598 working --is_my_input_or_output --is_my_input --is_my_output etc 2020-03-20 19:08:47 -04:00
Lex Berezhny af2f2282c2 txo_list returns txo funded by my account but sent to external address 2020-03-20 19:08:47 -04:00
Jack Robison 869a76c9bb Merge pull request #2851 from lbryio/transaction-info
Add `blockchain.transaction.info` and `blockchain.transaction.get_batch` to the wallet server for more efficient transaction fetching
2020-03-20 10:25:45 -04:00
Jack Robison 44b2964a6a add blockchain.transaction.get_batch 2020-03-19 19:20:30 -04:00
Jack Robison 8fddb57e0a reduce block lru cache sizes 2020-03-19 19:20:30 -04:00
Jack Robison c7a5a0cab0 add blockchain.transaction.info 2020-03-19 19:20:30 -04:00
Jack Robison c13aab3ffc Merge pull request #2823 from lbryio/multiple-db-readers
Use multiple processes for querying the db for api calls
2020-03-19 19:19:53 -04:00
Jack Robison 95a9a76598 windows 2020-03-19 19:19:02 -04:00
Jack Robison 923cfa3d50 another 2020-03-19 19:19:02 -04:00
Jack Robison 2c489168c2 feedback 2020-03-19 19:19:02 -04:00
Jack Robison f3292b4d34 feedback 2020-03-19 19:19:02 -04:00
Jack Robison 9769829b72 fix android executor 2020-03-19 19:19:02 -04:00
Jack Robison 36243d15cc fix peer_list 2020-03-19 19:19:02 -04:00
Jack Robison f4645f570c more read only calls 2020-03-19 19:19:02 -04:00
Jack Robison 61603ccfce write lock 2020-03-19 19:19:02 -04:00
Jack Robison a26cfc639c WAL 2020-03-19 19:19:02 -04:00
Jack Robison 90602931d8 multiple readers for transaction_list 2020-03-19 19:19:02 -04:00
Jack Robison d1b330028c multiple db reader processeses 2020-03-19 19:19:02 -04:00
Jack Robison 7a6b1930bf rename executor -> writer_executor 2020-03-19 19:19:02 -04:00
Lex Berezhny c271361552 Merge pull request #2865 from lbryio/lbrycrd-1744
upgrade lbrycrd to v0.17.4.4
2020-03-19 17:54:26 -04:00
Lex Berezhny 17789bc814 Merge pull request #2866 from lbryio/dont_shlex_windows
fixed issue with shell parsing on Windows
2020-03-19 17:53:47 -04:00
Brannon King d7c16e161a fixed issue with shell parsing on Windows 2020-03-19 13:11:57 -06:00
Lex Berezhny e71c17d7e7 upgrade lbrycrd to v0.17.4.4 2020-03-19 12:54:19 -04:00
Lex Berezhny 17d87eb157 Merge pull request #2861 from lbryio/ffmpeg_fixes
transcode in stream_update, re-enable transcode on Windows
2020-03-19 12:20:47 -04:00
Brannon King bf11bcc084 fixed transcoding tests for non-async execute
status ordering broke path check
2020-03-18 12:43:27 -06:00
Brannon King bd291109df addressing code review comments 2020-03-18 12:00:52 -06:00
Brannon King ac89ba9b8d don't require ProactorEventLoop on Windows
fix linter errors
2020-03-18 12:00:52 -06:00
Brannon King 5ab634e375 support search path for ffmpeg 2020-03-18 12:00:52 -06:00
Brannon King bb1978d976 ffmpeg now invoked via stream update 2020-03-18 12:00:52 -06:00
Lex Berezhny bea94ce8ac Merge pull request #2854 from lbryio/tracemalloc_api
Adds tracemalloc api for memory troubleshooting
2020-03-16 22:15:41 -04:00
Victor Shyba 9561f93594 disable tracemalloc after test finishes so it doesnt slow down everything else 2020-03-16 11:29:16 -03:00
Victor Shyba d44d5c3304 enable/disable instead of set 2020-03-16 06:40:22 -03:00
Victor Shyba ec541e2057 lint and improve tests 2020-03-11 19:56:37 -03:00
Victor Shyba 8169bf6b97 top objects api 2020-03-11 19:56:37 -03:00
Victor Shyba 56c8ad1221 start/stop tracemalloc over api 2020-03-11 19:56:37 -03:00
Lex Berezhny 6814f2e38c Merge pull request #2857 from eggplantbren/master
Trending algorithm with time delay and variable decay rate
2020-03-11 18:34:52 -04:00
Brendon J. Brewer 5f043b9a78 variable decay 2020-03-12 10:33:15 +13:00
Lex Berezhny c9092cd1c7 v0.64.0 2020-03-09 13:32:19 -04:00
Lex Berezhny 86cc65d894 Merge pull request #2848 from lbryio/claim_transaction_list_filtering
added new `txo_list` command with filtering for `--claim_id`, claim `--name` and `--is_received`/`--is_not_received`, also commands `claim_list`/`stream_list`/`channel_list`/`support_list` are based on `txo_list` and thus support most of the new filters
2020-03-08 23:33:38 -04:00
Lex Berezhny 8ef2647fa9 is_received 2020-03-08 23:11:03 -04:00
Lex Berezhny 3ff9e99416 added txo_list command 2020-03-08 16:22:49 -04:00
Lex Berezhny 1731046011 Merge pull request #2846 from lbryio/txo_is_spent
added `is_spent` attribute to transaction outputs
2020-03-06 21:10:30 -05:00
Lex Berezhny c3d96184b6 fix test 2020-03-06 20:35:26 -05:00
Lex Berezhny c255c606a7 added is_spent attribute to transaction outputs 2020-03-06 20:12:38 -05:00
Jack Robison cc69faa1fd v0.63.0 2020-03-06 15:58:34 -05:00
Jack Robison 0f53cd86c8 Merge branch 'debug_find_ffmpeg' 2020-03-06 15:54:24 -05:00
Brannon King 212c8f188d only change LD vars if running from bundle
missed a word


moved is_running_from_bundle
2020-03-06 15:53:09 -05:00
Brannon King dec248adec repair env modified by pyinstaller
see https://pyinstaller.readthedocs.io/en/stable/runtime-information.html#ld-library-path-libpath-considerations
2020-03-06 15:53:09 -05:00
Brannon King 98e6a066f4 add unit test and debug output 2020-03-06 15:53:09 -05:00
Jack Robison c90db54a3d Revert "v0.63.0"
This reverts commit 927d1569
2020-03-06 15:51:42 -05:00
Jack Robison 927d156933 v0.63.0 2020-03-06 14:49:53 -05:00
Jack Robison c71f91073f Merge pull request #2845 from lbryio/temp-disable-windows-ffmpeg
fix DHT setup and disable ffmpeg on windows temporarily
2020-03-06 14:20:18 -05:00
Jack Robison ab28387692 remove ProactorEventLoop and disable ffmpeg on windows temporarily 2020-03-06 13:45:34 -05:00
Lex Berezhny c4905d02b9 Merge pull request #2793 from lbryio/dont_validate_non_video
don't validate and repair files that aren't videos; add maximum video bit rate setting
2020-03-04 09:16:37 -05:00
Brannon King ee39880fb5 fix items from review 2020-03-03 21:38:30 -07:00
Brannon King 926b3e56b9 take a hint, lint 2020-03-03 21:38:30 -07:00
Brannon King e060df5367 hide ValueError 2020-03-03 21:38:30 -07:00
Brannon King a3294d4a0d make bit_rate check support maximum 2020-03-03 21:38:30 -07:00
Brannon King 19ce0ab246 ogg -> ogv, ignore files that aren't video 2020-03-03 21:38:30 -07:00
Lex Berezhny 75a1cc0d33 Merge pull request #2840 from lbryio/status_returns_server_features
`status` command returns features available of currently connected wallet server, including trending algorithm
2020-03-03 20:47:04 -05:00
Lex Berezhny 05228529b0 fix tests 2020-03-03 20:26:07 -05:00
Lex Berezhny 3cbeadfbc3 lbrynet status command returns features available of currenty connected wallet server, including thetrending algorithm used 2020-03-03 20:04:57 -05:00
Lex Berezhny 3d4938c0e2 Merge pull request #2839 from lbryio/file_analysis_join_path
use `os.path.join()` instead of string addition when searching for file analysis binaries
2020-03-03 10:48:06 -05:00
Lex Berezhny 1b850b8a2b dont prepend path when doing which ffmpeg 2020-03-02 21:11:11 -05:00
Lex Berezhny ebd33f1869 use os.path.join instead of string addition when searching for file analysis binaries 2020-03-02 20:34:54 -05:00
Jack Robison ac846b4df3 v0.62.0 2020-03-01 20:29:32 -05:00
Jack Robison 0d427c9b90 Merge pull request #2836 from lbryio/wallet-server-lru-caches
Add LRU caches for blocks and block hashes in the wallet server
2020-03-01 15:19:28 -05:00
Jack Robison dc6194f862 perf counter 2020-03-01 15:17:21 -05:00
Jack Robison de50214e1f block lru cache 2020-03-01 15:17:21 -05:00
Jack Robison 5ec4a88c35 block hash lru cache 2020-03-01 15:17:05 -05:00
Jack Robison 31c141e757 Merge pull request #2835 from lbryio/wallet-server-threadpools
Improve wallet server performance with separate thread pools for leveldb, the block processor, and mempool
2020-03-01 15:15:20 -05:00
Jack Robison 9d5760d899 fix test 2020-03-01 14:41:14 -05:00
Jack Robison 7945e1ea3c mempool threadpool 2020-03-01 12:40:13 -05:00
Jack Robison d94c40e371 leveldb threadpool 2020-03-01 12:40:13 -05:00
Jack Robison 31f22122e8 block processor threadpool 2020-03-01 12:40:13 -05:00
Lex Berezhny 506582aa2b Merge pull request #2834 from lbryio/list_resolve_includes_local_data
when using `--resolve` with local claim `*_list` commands, update resolved result with local metadata (such as `is_mine`)
2020-02-29 15:48:49 -05:00
Lex Berezhny a02b251c9b when using --resolve with local claim list commands, update resolved result with local metadata (such as is_mine and is_mine) 2020-02-29 09:42:11 -05:00
Lex Berezhny 34cbb6fa79 Merge pull request #2831 from lbryio/claim_type_index_improvement
update `claim_type_*` db indexes to be more performant
2020-02-27 14:27:24 -05:00
Lex Berezhny 33679b56fd update claim_type_* indexes to be more peformant 2020-02-27 12:07:38 -05:00
Lex Berezhny 8897c4d560 Merge pull request #2816 from osilkin98/removes-anonymous
removes ability to comment anonymously
2020-02-27 10:37:47 -05:00
Oleg Silkin f0c18ccbe7 Removes testing for anonymous comments, fixes up existing asserts 2020-02-26 19:59:38 -05:00
Oleg Silkin e560d83c51 drops support for commenting without a channel 2020-02-26 19:58:03 -05:00
Lex Berezhny eca7addc67 v0.61.0 2020-02-24 10:01:03 -05:00
Lex Berezhny 263f3ba5c9 Merge pull request #2828 from lbryio/update_lbrycrd
update lbrycrd in tests to v0.17.4.3
2020-02-22 14:47:48 -05:00
Lex Berezhny f514123ef0 update lbrycrd 2020-02-22 13:42:28 -05:00
Lex Berezhny 9f461db0d0 Merge pull request #2827 from lbryio/repost_inherit_tags
reposts inherit the tags of the original claim, filtered/blocked claims cause their reposts to also be filtered/blocked
2020-02-22 13:04:31 -05:00
Lex Berezhny 7de8670616 blocked original repost causes all reposts to also be blocked 2020-02-22 12:49:20 -05:00
Lex Berezhny 8c7908b200 reposts inherit the tags of the original claim 2020-02-22 12:23:11 -05:00
Lex Berezhny e7722e039f Merge pull request #2826 from lbryio/fix_not_channel_ids
`claim_search --not_channel_ids` excludes the channel itself and not just claims in the channel
2020-02-21 23:01:00 -05:00
Lex Berezhny 039bc0208a fix tests 2020-02-21 22:22:28 -05:00
Lex Berezhny d57900a069 claim_search --not_channel_ids excludes the channel itself and not just claims in the channel 2020-02-21 21:59:46 -05:00
Jack Robison c1153302aa v0.60.1 2020-02-21 16:13:30 -05:00
Victor Shyba 1086a3297f check its broadcasted and use a retriable call 2020-02-21 17:51:46 -03:00
Jack Robison 5c613934ca Revert "v0.60.1"
This reverts commit 7f03b13579.
2020-02-21 15:05:33 -05:00
Jack Robison 6d7e9092f8 Merge pull request #2825 from lbryio/tx_from_stream_controller
use the stream controller tx instead of local history
2020-02-21 15:04:57 -05:00
Victor Shyba 8bb6f328dc use the stream controller tx instead of local history 2020-02-21 16:41:23 -03:00
Jack Robison 7f03b13579 v0.60.1 2020-02-21 14:14:15 -05:00
Jack Robison 09008ea991 Merge pull request #2824 from lbryio/skip_wallet_server_payment
max wallet server payment fee defaults to 0 and component doesnt start when its not a positive value
2020-02-21 13:27:58 -05:00
Jack Robison ee234212e6 Merge branch 'master' into skip_wallet_server_payment 2020-02-21 13:26:38 -05:00
Jack Robison 277b243f52 Merge pull request #2772 from lbryio/reflector-status
Add `is_fully_reflected` to `file_list` response
2020-02-21 13:21:36 -05:00
Jack Robison 0973ac753f add is_fully_reflected to file_list response 2020-02-21 12:49:19 -05:00
Victor Shyba f41cfbfb97 max wallet server payment fee defaults to 0 and component doesnt start in that case 2020-02-21 14:47:09 -03:00
Lex Berezhny a70980c81a reverting v0.60.1 release 2020-02-21 12:43:43 -05:00
Lex Berezhny 12a962f656 Merge pull request #2822 from lbryio/stream_controller_payment
use stream controller and documented errors for wallet server payments
2020-02-21 12:34:07 -05:00
Lex Berezhny 5936444f3e more reliable wait for wallet server payment test 2020-02-21 12:18:44 -05:00
Lex Berezhny c7d42f00c6 shorten error message for ServerPaymentFeeAboveMaxAllowed 2020-02-21 10:16:56 -05:00
Lex Berezhny b1318a9958 minor cleanup 2020-02-21 10:05:46 -05:00
Victor Shyba d2560d260c use stream controller and documented errors 2020-02-21 00:04:37 -03:00
Jack Robison f40a61cf9a v0.60.1 2020-02-20 17:56:54 -05:00
Jack Robison e0623578bf Merge pull request #2821 from lbryio/ffmpeg-find
add `ffmpeg_find` api to check ffmpeg installation status, don't recheck for ffmpeg in `status`
2020-02-20 17:48:36 -05:00
Jack Robison ead8daaa14 doc 2020-02-20 17:27:18 -05:00
Jack Robison 23b4b9e230 add ffmpeg_find api, don't recheck for it in status 2020-02-20 16:43:41 -05:00
Jack Robison 947017e334 Merge pull request #2820 from lbryio/disconnect-on-oversized
disconnect from client sending message larger than MAX_RECEIVE
2020-02-20 16:24:26 -05:00
Jack Robison 7fd0d6507f disconnect from client sending message larger than MAX_RECEIVE 2020-02-20 16:08:21 -05:00
Jack Robison ad1e9ef086 Merge pull request #2819 from lbryio/max-receive
add MAX_RECEIVE setting to wallet server
2020-02-20 14:38:11 -05:00
Jack Robison 2f1d08e417 add MAX_RECEIVE wallet server setting 2020-02-20 14:11:16 -05:00
Lex Berezhny 55f4eb80ba Merge pull request #2815 from lbryio/apply_share_usage_data_live
apply share_usage_data as its set, without restarting
2020-02-20 13:35:47 -05:00
Victor Shyba 7c160ff65e remove unused parameters 2020-02-20 14:38:13 -03:00
Victor Shyba fc5d5faaed use conf directly isntead of lambda 2020-02-20 14:27:39 -03:00
Victor Shyba 5394f1763c simplify external ip logic 2020-02-20 12:12:40 -03:00
Victor Shyba 050b67c9d6 apply share_usage_data as its set 2020-02-20 12:12:40 -03:00
Lex Berezhny 4b87cb45ee Merge pull request #2707 from lbryio/pay_wallet_servers
wallet servers requiring a daily fee will automatically get paid by client
2020-02-18 17:40:23 -05:00
Victor Shyba f0e1db319c make wallet server payments a component 2020-02-18 19:18:30 -03:00
Victor Shyba 3950715237 add max_wallet_server_fee conf to limit daily wallet server payments 2020-02-18 18:10:58 -03:00
Victor Shyba ae9ba14b59 use both donation and payment addresses separately 2020-02-18 18:10:58 -03:00
Victor Shyba ad6c6fbe35 fixes from review and add analytics 2020-02-18 18:10:58 -03:00
Victor Shyba ffa5c20c88 fix hanging test 2020-02-18 18:10:58 -03:00
Victor Shyba 5a6218eeca improve tests, fix types 2020-02-18 18:10:58 -03:00
Victor Shyba 200531dd96 wait for tx instead 2020-02-18 18:10:58 -03:00
Victor Shyba 15a2f048ac pay wallet server test and initial prototype 2020-02-18 18:10:58 -03:00
Victor Shyba d317a4042c change daily_fee to be lbc instead of dewies 2020-02-18 18:10:58 -03:00
Victor Shyba c2d717aba5 old code from #2683 2020-02-18 18:10:58 -03:00
Lex Berezhny ef74777df1 v0.60.0 2020-02-17 17:52:11 -05:00
Lex Berezhny f6dbf99fb5 Merge pull request #2811 from lbryio/index_fixes
wallet server indexes
2020-02-17 17:43:55 -05:00
Lex Berezhny 0b2d9e15b4 added order bys for fee_amount 2020-02-17 17:24:48 -05:00
Lex Berezhny d14e5e75e8 all three order bys for duration filter 2020-02-17 17:24:48 -05:00
Lex Berezhny df11ef34d2 index cleanup 2020-02-17 17:24:48 -05:00
Lex Berezhny 4fc619262c replacing indexes on trending_global, trending_mixed to be trending_group, trending_mixed 2020-02-17 17:24:48 -05:00
Lex Berezhny 2750eee86d Merge pull request #2810 from lbryio/no_default_order_by
removed default `order_by` for `claim_search`
2020-02-17 17:04:41 -05:00
Lex Berezhny 71cb03345b removed default order_by for claim_search 2020-02-17 16:26:29 -05:00
Lex Berezhny 22c8ca6c40 Merge pull request #2806 from lbryio/privacyDefault
change default privacy setting
2020-02-17 16:21:34 -05:00
Victor Shyba 237a8965eb fix tests 2020-02-17 18:00:23 -03:00
jessop 7d872c7863 change default privacy setting 2020-02-13 18:56:11 -05:00
Jack Robison 0d08858dfb v0.59.2 2020-02-13 17:22:21 -05:00
Jack Robison 5c6d63c93d Merge pull request #2803 from lbryio/fix-2802
fix claim apis failing with --resolve if no claims are present
2020-02-13 15:15:56 -05:00
Jack Robison c8c10d2bb0 fix claim apis failing with --resolve if no claims are present
fixes https://github.com/lbryio/lbry-sdk/issues/2802
2020-02-13 14:58:51 -05:00
Jack Robison f4d3e9ea6a add development wallet server deploy script 2020-02-13 13:54:46 -05:00
Jack Robison 3713d3488d Merge pull request #2800 from lbryio/fix-2746
Fix race condition setting/clearing the claim_search and resolve cache
2020-02-13 11:39:27 -05:00
Jack Robison 9b46d03c91 fix race between setting items in and clearing the resolve/claim_search cache 2020-02-13 11:17:21 -05:00
Jack Robison fe4e738580 Merge pull request #2798 from lbryio/repost-index
Add sqlite index for reposted claims
2020-02-13 10:33:36 -05:00
Jack Robison 205b0c4263 call get_platform only once 2020-02-13 10:04:50 -05:00
Jack Robison 78eef25f1a add reposted_claim_hash index
fixes https://github.com/lbryio/lbry-sdk/issues/2782
2020-02-12 17:39:57 -05:00
Lex Berezhny c9305fd070 Merge pull request #2796 from lbryio/xenial_build
linux build of LBRY SDK for Ubuntu 16.04 Xenial
2020-02-12 14:28:41 -05:00
Lex Berezhny 20774280b9 Merge pull request #2790 from lbryio/sql_in_for_single_value
SQL generation fix to handle IN operation for one value lists
2020-02-12 14:27:36 -05:00
Lex Berezhny 74bd2557af linux build of LBRY SDK for Ubuntu 16.05 Xenial 2020-02-12 14:20:06 -05:00
Lex Berezhny 605fc8ecd8 claim_search by --claim_type=repost support 2020-02-12 11:51:35 -05:00
Lex Berezhny 6b745c53dc make sure SQL IN works with python set 2020-02-12 11:41:32 -05:00
Lex Berezhny 98c60b38fc disable too-many-nested-blocks lint rule 2020-02-12 10:47:40 -05:00
Lex Berezhny be4515ec9a use sets for __in constraint values 2020-02-12 10:39:25 -05:00
Lex Berezhny dcb1b64696 SQL generation fix to handle IN operation for one value lists 2020-02-12 10:31:27 -05:00
Jack Robison 1190a60c7a v0.59.1 2020-02-11 16:56:37 -05:00
Jack Robison 82ecf207c6 Merge pull request #2789 from lbryio/update-default-servers
update default wallet servers
2020-02-11 16:36:31 -05:00
Jack Robison 84c7f1bf58 update test 2020-02-11 16:13:55 -05:00
Jack Robison 5ac453fe07 update default wallet servers 2020-02-11 13:51:15 -05:00
Lex Berezhny c2f39d653d v0.59.0 2020-02-10 12:08:24 -05:00
Lex Berezhny 748178f536 Merge pull request #2780 from lbryio/release_current
manually incremented version and added support to release script to r…
2020-02-09 20:41:00 -05:00
Lex Berezhny ffbef69dc7 manually incremented version and added support to release script to release current version 2020-02-09 20:26:15 -05:00
Lex Berezhny 3757124323 Merge pull request #2771 from lbryio/update_url_regex
update url regex to include missing cases
2020-02-09 18:17:35 -05:00
Victor Shyba e65f214b3c fix test using space in name 2020-02-09 17:55:29 -05:00
Victor Shyba f14004e56b update url regex to cases tom reported 2020-02-09 17:55:29 -05:00
Lex Berezhny dadc004dd6 Merge pull request #2765 from eggplantbren/master
fixed some issues with AR trending
2020-02-09 17:53:27 -05:00
Brendon J. Brewer 3697d9e1f0 Tweaked some parameters, back to 134 2020-02-09 17:38:34 -05:00
Brendon J. Brewer 94524f7330 lint 2020-02-09 17:38:34 -05:00
Brendon J. Brewer 17d8a3e5d6 Use claim_hash not claim_id 2020-02-09 17:38:24 -05:00
Brendon J. Brewer 39996d7612 Only create file during install if logging is turned on 2020-02-09 17:37:55 -05:00
Lex Berezhny 601ee4320f Merge pull request #2768 from mkroman/master
update cffi to v1.13.2
2020-02-09 17:35:06 -05:00
Mikkel Kroman 6e4f6fa92d Update cffi to v1.13.2
This fixes compilation issues for CPython 3.8+
2020-02-09 17:19:53 -05:00
Lex Berezhny b7103c29dd Merge pull request #2773 from lbryio/fix_video_analysis_error
fix video analysis error
2020-02-09 17:19:23 -05:00
Brannon King e53209e5ae handle early lbrycrd exit in tests 2020-02-09 17:02:16 -05:00
Brannon King 501fbd3114 handle strange file metadata 2020-02-09 17:02:16 -05:00
Lex Berezhny 5626f43e2b Merge pull request #2779 from lbryio/github_actions
build community PRs using GitHub Actions
2020-02-09 16:49:40 -05:00
Lex Berezhny 0185cb48fb run lint and tests in parallel 2020-02-09 16:34:04 -05:00
Lex Berezhny 9a49eb06da fix import path in test_transcoding.py 2020-02-09 16:15:20 -05:00
Lex Berezhny 3fbc012231 moved blockchain/test_transcoding.py -> other/test_transcoding.py 2020-02-09 16:15:20 -05:00
Lex Berezhny 3938cf33d8 take github actions for a spin 2020-02-09 16:15:04 -05:00
Lex Berezhny 38b3d0c87c Merge pull request #2777 from lbryio/claim_list_multi_type
`claim_list` argument `--claim_type` can now be repeated to pass a list of claim_types
2020-02-08 23:34:34 -05:00
Lex Berezhny 15a2fa6199 claim_list --claim_type argument can be repeated 2020-02-08 23:34:04 -05:00
Lex Berezhny e834209d40 Merge pull request #2776 from lbryio/resolve_includes_censoring_channels_url
`resolve` results which are censored now include full details of channel which did the censoring
2020-02-08 23:32:44 -05:00
Lex Berezhny 9607d21828 censored searches/resolves include metadata of channel which did the censoring 2020-02-07 18:50:29 -05:00
Lex Berezhny 6fbbf36143 pylint 2020-02-07 18:44:30 -05:00
Lex Berezhny db25f3282e lbry.error: save arguments on self and support for functions around arguments 2020-02-07 18:25:06 -05:00
Jack Robison ceb92838e7 v0.58.2 2020-02-06 11:37:09 -05:00
Jack Robison afb98f2cfc Merge pull request #2770 from lbryio/disable-save-content-claim
Add `save_resolved_claims` config setting for better performance when not using file_x commands
2020-02-06 11:32:49 -05:00
Jack Robison ed785fb087 add save_resolved_claims config setting to disable saving claims upon resolving them 2020-02-06 11:27:26 -05:00
Jack Robison 3921180d77 v0.58.1 2020-02-04 15:12:55 -05:00
Jack Robison ef36e0311e Merge pull request #2764 from lbryio/version-analytics
Add version tracking to prometheus metrics
2020-02-04 12:44:30 -05:00
Jack Robison bf4cbe1204 add version to session count metric 2020-02-04 12:18:32 -05:00
Jack Robison 28eee4da25 add version metrics to prometheus 2020-02-04 10:52:08 -05:00
Jack Robison 933ccf6deb add block updates to prometheus 2020-02-04 10:26:22 -05:00
Jack Robison 9ac41322e5 add cpu count to prometheus 2020-02-04 10:25:40 -05:00
Jack Robison c3233e03ef Merge pull request #2762 from lbryio/fix-2368
cancel reflector uploads upon file delete
2020-02-03 23:51:12 -05:00
Jack Robison 111871bb28 update test 2020-02-03 23:00:45 -05:00
Jack Robison bf5b5f43e3 non blocking analytics 2020-02-03 23:00:45 -05:00
Jack Robison 2ed8ebff09 handle ConnectionError and ValueError in blob sendfile 2020-02-03 23:00:45 -05:00
Jack Robison 34eb856d09 cancel reflector uploads upon file delete
-remove unnecessary db call in stream_update
2020-02-03 23:00:45 -05:00
Lex Berezhny 2d644bdfb0 v0.58.0 2020-02-03 22:33:03 -05:00
Lex Berezhny 741e5cf103 regenerated api docs 2020-02-03 22:30:39 -05:00
Lex Berezhny d745c04fe6 Merge pull request #2726 from lbryio/add_video_transcoding
added video file validation and optimization when publishing streams (using `--validate_file` and `--optimize_file` arguments)
2020-02-03 22:29:15 -05:00
Lex Berezhny 3a3c63956a removed unnecessary try/finally 2020-02-03 22:06:15 -05:00
Lex Berezhny 153bdf576a removed wheel from setup.py 2020-02-03 22:05:47 -05:00
Lex Berezhny 6525ee6510 moved __init__ to top of class 2020-02-03 22:05:23 -05:00
Brannon King 1780ddd329 added ffmpeg status, addressed items from code review
linter
2020-02-03 21:53:42 -05:00
Brannon King 85ad972ca8 return string instead of path object for compatibility with hachoir 2020-02-03 21:53:42 -05:00
Brannon King a90b60799a Fixed check_video.py on Windows
using a cross-platform workaround


fixed proactor use in the SDK


fixed linter
2020-02-03 21:53:42 -05:00
Brannon King 47e8f74da9 changed to list append, relative claim test 2020-02-03 21:53:42 -05:00
Brannon King fac28072ab added unit tests, other minor fixes
added universe


try again


try 4
2020-02-03 21:53:42 -05:00
Brannon King ab77541f36 in progress on video transcoding
works


Fixing lint tests


remove eval
2020-02-03 21:53:42 -05:00
Lex Berezhny 9eb769f340 Merge pull request #2760 from lbryio/channel_hash_wallet_server_idx
added database index for several `channel_hash` based queries
2020-02-03 20:56:48 -05:00
Lex Berezhny 1e81d6f848 added database index for various channel_hash based queries 2020-02-03 20:28:39 -05:00
Jack Robison dc393f4b77 Merge pull request #2761 from lbryio/catch-notification-timeout
catch TimeoutError when attempting to send notifications
2020-02-03 19:03:07 -05:00
Jack Robison dd1de530c3 catch TimeoutError when attempting to send notifications 2020-02-03 18:46:17 -05:00
Lex Berezhny 74cd887249 v0.57.0 2020-02-03 13:41:33 -05:00
Lex Berezhny b40bb35652 change MINIMUM_REQUIRED on client for servers it connects to 2020-02-03 13:41:05 -05:00
Lex Berezhny dfe257af86 Revert "test to make sure next release tests pass"
This reverts commit ebb2a09107.
2020-02-03 13:39:36 -05:00
Lex Berezhny ebb2a09107 test to make sure next release tests pass 2020-02-03 13:22:33 -05:00
Lex Berezhny c157909b4e Merge pull request #2758 from ykris45/patch-6
update copyright year in license
2020-02-02 20:19:33 -05:00
YULIUS KURNIAWAN KRISTIANTO a744239b22 update year 2020-02-03 04:04:51 +07:00
Lex Berezhny 9b4417be87 Merge pull request #2757 from lbryio/resolve_for_local_list
added `--resolve` to local `claim_list`/`stream_list`/`channel_list` commands which returns more metadata for local claims
2020-02-01 18:15:35 -05:00
Lex Berezhny 84f807e278 regenerate docs 2020-02-01 17:59:57 -05:00
Lex Berezhny 448635a945 added --resolve to local *_list commands 2020-02-01 17:59:10 -05:00
Lex Berezhny 638e3e6b3d Merge pull request #2756 from lbryio/blocked_resolve
resolve errors make distinction between truely not found claims and claims which were censored by wallet server
2020-02-01 14:00:37 -05:00
Lex Berezhny bfe711bd42 fixing tests 2020-02-01 13:29:55 -05:00
Lex Berezhny d5eed91e7f fix sql unit tests 2020-02-01 13:03:39 -05:00
Lex Berezhny 15abf49211 censored resolve responses return appropriate error 2020-02-01 12:53:39 -05:00
Lex Berezhny b7eec0586c added ResolveCensoredError 2020-02-01 12:49:01 -05:00
Lex Berezhny e639124a69 Merge pull request #2755 from eggplantbren/master
fixed initialization problem with ar trending algorithm
2020-01-30 13:44:25 -05:00
Brendon J. Brewer 22bca7a16e Length zero could also just mean there aren't any claims 2020-01-31 07:34:43 +13:00
Lex Berezhny 6f66b354e5 Merge pull request #2754 from lbryio/feat-claim_search_duration
add duration to claim search params
2020-01-29 16:43:34 -05:00
jessop f98e3320ae make review changes 2020-01-29 16:25:04 -05:00
jessop 73c958222b add duration to claim search params 2020-01-29 15:46:47 -05:00
Lex Berezhny e43230e46a v0.56.0 2020-01-28 19:58:28 -05:00
Lex Berezhny dd07bc29eb Merge pull request #2747 from eggplantbren/master
new trending algorithm contributed by Brendon Brewer, to enable on wallet server set `TRENDING_ALGORITHMS` environment variable on wallet server to `ar`
2020-01-28 19:36:52 -05:00
Lex Berezhny 790bbfcc99 integrate ar trending algorithm into configurable trending system 2020-01-28 18:11:32 -05:00
Brendon J. Brewer 994e70d43a ar trending algorithm 2020-01-28 18:11:32 -05:00
Jack Robison 7d5e19716a Merge pull request #2749 from lbryio/prometheus-interrupts
add metrics to wallet server prometheus
2020-01-28 16:31:10 -05:00
Jack Robison 11530c675d initialize_request_handlers, remove local rpc handlers 2020-01-28 15:54:22 -05:00
Jack Robison 1c474352fe add wallet server prometheus metrics
-sessions gauge
-requests counter
-response times histogram
-notification counter
-request errors counter
-interrupt counter
-operational error counter
-internal error counter
-reader executor times histogram
-pending queries gauge
-lbrycrd request counter
-client versions counter
2020-01-28 15:54:22 -05:00
Jack Robison 3b2d635390 Revert "Revert "configurable trending algorithms""
This reverts commit 55e08f96a6.
2020-01-28 15:53:57 -05:00
Jack Robison 055c532d31 Merge pull request #2752 from lbryio/revert-2750-configurable_trending
Revert "configurable trending algorithms"
2020-01-28 15:04:29 -05:00
Jack Robison 55e08f96a6 Revert "configurable trending algorithms" 2020-01-28 15:03:09 -05:00
Lex Berezhny 3e0a9180bc Merge pull request #2750 from lbryio/configurable_trending
configurable trending algorithms
2020-01-28 12:30:26 -05:00
Lex Berezhny 01ed71b243 configurable trending 2020-01-28 11:58:27 -05:00
Lex Berezhny 595c058517 moved trending.py to trending/zscore.py 2020-01-28 11:58:27 -05:00
Alex Grintsvayg e04b0a6995 add docker tag to prometheus version info 2020-01-27 14:16:18 -05:00
Jack Robison 8961848ab1 Merge pull request #2748 from lbryio/cached-wallet-id
faster wallet.id attribute, improves speed of selecting wallets during api calls
2020-01-27 13:58:23 -05:00
Jack Robison 6fb1c72b7d cached wallet id 2020-01-27 13:57:21 -05:00
Alex Grintsvayg 400d79d6ab prom port off by default 2020-01-27 13:55:44 -05:00
Lex Berezhny ce6018f387 v0.55.0 2020-01-27 12:20:41 -05:00
Alex Grintsvayg 65d9dca917 Merge branch 'docker'
* docker:
  fix gitlab ci
  add min_version and method name to metrics
  include docker tag in build info
  build_type.py -> build_info.py
  add docker build info to wallet server image
2020-01-23 13:13:33 -05:00
Alex Grintsvayg 2232b08351 fix gitlab ci 2020-01-23 13:13:20 -05:00
Alex Grintsvayg 0732ab6ad5 add min_version and method name to metrics 2020-01-23 13:13:20 -05:00
Alex Grintsvayg fb568768c5 include docker tag in build info 2020-01-23 13:13:19 -05:00
Alex Grintsvayg 7fd56e0add build_type.py -> build_info.py 2020-01-23 13:13:19 -05:00
Alex Grintsvayg 38a3f8cf4c add docker build info to wallet server image 2020-01-23 13:13:19 -05:00
Lex Berezhny 8d8a5b36b6 Merge pull request #2737 from lbryio/fix_repost_resolve_bug
fix bug where repost claims were not being fully resolved
2020-01-22 23:32:55 -05:00
Lex Berezhny a70cc7beb1 fix bug where repost claims were not being fully resolved 2020-01-22 22:59:05 -05:00
Alex Grintsvayg c5421699b1 add prometheus port to docker-compose.yml 2020-01-22 20:08:29 -05:00
Alex Grintsvayg d9f809864f include version and build info in prometheus output 2020-01-22 19:13:58 -05:00
Lex Berezhny bb23f509d7 rename BLOCKING_CHANNELS_IDS/FILTERING_CHANNELS_IDS to remove double S 2020-01-22 11:02:01 -05:00
Alex Grin 254e184677 Merge pull request #2734 from lbryio/prom2
Add prometheus metrics collection to client and server (take two)
2020-01-22 10:28:54 -05:00
Alex Grintsvayg 6c07141abd Add prometheus metrics collection to client and server 2020-01-22 10:28:17 -05:00
Lex Berezhny 5822fc1d5b v0.54.1 2020-01-22 08:40:35 -05:00
Lex Berezhny 2cd7ea257c Added support to differentiate between filtering and blocking for content censoring 2020-01-22 08:39:14 -05:00
Alex Grin 0a21b72f9c v0.54.0 2020-01-21 16:17:12 -05:00
Alex Grintsvayg 80280f6f4a Revert "v0.54.0"
This reverts commit bd3b4906d1.
2020-01-21 15:34:35 -05:00
Alex Grintsvayg 810a8e76d1 Pin pip to 19.3.1 on Windows
pip 20 made the following change:

> Remove interpreter-specific major version tag e.g. cp3-none-any from consideration.
> https://github.com/pypa/pip/pull/7355

The coincurve pip package only has a major version tag for its windows
wheel. pip20 will not allow us to install that wheel, and compiling
coincurve from source causes errors. So we cannot use pip 20 or above
until coincurve fixes its compatibility tags, we can compile coincurve
from source, we stop using coincurve, or we figure out another way to
install it.

Note that on osx and linux, coincurve has the correct tags.
2020-01-21 15:24:29 -05:00
Lex Berezhny bd3b4906d1 v0.54.0 2020-01-21 11:30:14 -05:00
Lex Berezhny d9c740f014 update tests to use 0.54.0 minimum version 2020-01-21 11:28:56 -05:00
Alex Grintsvayg ec63a18960 Revert "v0.54.0"
This reverts commit fab0618b6b.
2020-01-21 11:23:51 -05:00
Alex Grin fab0618b6b v0.54.0 2020-01-21 11:00:38 -05:00
Alex Grintsvayg 13e57e7aa8 Revert "v0.54.0"
This reverts commit 836b66e110.
2020-01-21 10:43:21 -05:00
Alex Grintsvayg 1299c9162c Revert "Add prometheus metrics collection to client and server"
This reverts commit 59a5bacb2e.
2020-01-21 10:30:49 -05:00
Alex Grintsvayg a64f33dbcc ignore coverage files 2020-01-21 10:30:48 -05:00
Alex Grin 836b66e110 v0.54.0 2020-01-20 14:26:36 -05:00
Alex Grintsvayg 802119d789 Revert "v0.54.0"
This reverts commit aec4d1dc60.
2020-01-20 14:24:30 -05:00
Alex Grintsvayg f94135cadd make pycharm happy about long lines 2020-01-20 14:18:00 -05:00
Alex Grintsvayg 474c0f980e skip randomly-failing dht test 2020-01-20 14:16:52 -05:00
Alex Grintsvayg 2cf0f791f2 remove unused import 2020-01-20 13:44:49 -05:00
Alex Grin aec4d1dc60 v0.54.0 2020-01-20 13:13:30 -05:00
Alex Grintsvayg 8d28bb5a66 bump min protocol version 2020-01-20 12:53:10 -05:00
Alex Grintsvayg 1d5e553f9c Merge branch 'blocked_search_metadata'
* blocked_search_metadata:
  add index for claim_type and release_time
  fix test
  fix json api generator
  add pagination for claim_search
  regenerate protobufs
  using multiprocessing.Manager to keep blocked content synced between readers
2020-01-20 12:51:02 -05:00
Lex Berezhny 9d79c52d20 add index for claim_type and release_time 2020-01-20 12:50:20 -05:00
Lex Berezhny 1ac03e2f1d fix test 2020-01-20 12:50:16 -05:00
Lex Berezhny 801f05f45e fix json api generator 2020-01-20 12:50:16 -05:00
Lex Berezhny 345196aa3b add pagination for claim_search 2020-01-20 12:49:42 -05:00
Lex Berezhny a01b52421e regenerate protobufs 2020-01-20 12:49:41 -05:00
Lex Berezhny 86cedfe8b2 using multiprocessing.Manager to keep blocked content synced between readers 2020-01-20 12:49:41 -05:00
Alex Grintsvayg 709128225b Merge branch 'prometheus'
* prometheus:
  Add prometheus metrics collection to client and server
2020-01-20 12:45:11 -05:00
Alex Grintsvayg 59a5bacb2e Add prometheus metrics collection to client and server 2020-01-20 12:43:49 -05:00
Jack Robison 3a21df31ee v0.53.3 2020-01-16 16:43:26 -05:00
Jack Robison 0e3407ec2f Merge pull request #2730 from lbryio/bump-aioupnp2
update aioupnp requirement to 0.0.17
2020-01-16 16:00:42 -05:00
Jack Robison fd6609e961 bump aioupnp requirement to 0.0.17 2020-01-16 15:40:29 -05:00
Jack Robison b0b5e045ff non blocking aioupnp success analytics 2020-01-16 15:40:24 -05:00
Jack Robison f293d7cccc pin windows pyinstaller 2020-01-16 15:02:15 -05:00
Jack Robison 974ed29c36 Merge pull request #2729 from lbryio/fix-headers-subscription-to-old
disconnect from server that returns rpc error for server.version
2020-01-15 17:59:17 -05:00
Jack Robison 53eb033034 disconnect from server that returns rpc error for server.version
-the server returns this error if it thinks we are incompatible with it
2020-01-15 17:06:13 -05:00
Jack Robison f2545b98ab Merge pull request #2728 from lbryio/check-server-version-from-client
check minimum server version from wallet client
2020-01-15 16:53:10 -05:00
Jack Robison 1bf51e855d check minimum server version from wallet client 2020-01-15 15:55:39 -05:00
Jack Robison e4da2a695b v0.53.2 2020-01-14 16:47:56 -05:00
Lex Berezhny 1a905d9a42 make --start-tag work again in release/script.py 2020-01-13 21:40:07 -05:00
Victor Shyba 7d9d0c15d3 fix test_direct_sync flakiness 2020-01-13 21:17:10 -05:00
Victor Shyba d34f5c2712 fix None in history, check records after wait 2020-01-13 21:17:10 -05:00
Victor Shyba a56dd66c98 separate cases and make it deterministic 2020-01-13 21:17:10 -05:00
Victor Shyba 3946bc6662 fix and document new edge case on tests 2020-01-13 21:17:10 -05:00
Victor Shyba 6647dd8f08 fix decrypting invalid bytes with valid padding 2020-01-13 21:17:10 -05:00
Jack Robison 9371122bed Merge pull request #2725 from lbryio/stream-manager-startup
faster stream manager startup
2020-01-13 18:09:49 -05:00
Jack Robison 734cd8ee08 logging 2020-01-13 17:51:05 -05:00
Jack Robison d85d9d05bb fast get_all_lbry_files 2020-01-13 17:51:05 -05:00
Jack Robison 1d7b87b3a9 fix inefficient loop in update_manually_removed_files_since_last_run 2020-01-13 17:51:02 -05:00
Lex Berezhny f0cfde36f2 v0.53.1 2020-01-13 14:12:30 -05:00
Lex Berezhny 7e220d2741 downgrade setuptools 2020-01-13 14:03:45 -05:00
Lex Berezhny c4b1351a43 stop building on travis 2020-01-13 14:03:45 -05:00
Lex Berezhny 90d2597bc3 pin to pyinstaller==3.5 2020-01-13 14:03:45 -05:00
Lex Berezhny c1508c94c2 v0.53.0 2020-01-13 12:23:41 -05:00
Jack Robison 1a0d805bad Merge pull request #2720 from lbryio/loggly-database-error
fix database error caused by logging tracebacks from within the sqlite transaction runner
2020-01-12 16:23:26 -05:00
Jack Robison 1a802469f5 missing fetchall() 2020-01-12 16:16:48 -05:00
Jack Robison 646ae7e2d1 catch RuntimeError in loggly handler to handle cases where there is not an event loop 2020-01-12 16:16:48 -05:00
Jack Robison 563d76ff9f Merge pull request #2719 from lbryio/refactor_header_progress
refactor header progress calculation, making it simpler to understand and not prone to division by zero
2020-01-12 16:16:23 -05:00
Victor Shyba 54d0473e85 refactor header progress 2020-01-12 00:56:43 -03:00
Jack Robison ec8a61c3af Merge pull request #2718 from lbryio/add-connected-to-wallet-status
add `connected` server to wallet status
2020-01-11 01:11:34 -05:00
Jack Robison 0315eaff8a add connected server to wallet status 2020-01-11 00:33:52 -05:00
Jack Robison b34f44c867 logging 2020-01-11 00:17:32 -05:00
Jack Robison ea7056835f Merge pull request #2716 from lbryio/batched-address-subscriptions
speed up wallet sync and startup by batching address history subscriptions
2020-01-10 21:32:12 -05:00
Jack Robison 0bb4cdadd9 use network.subscribe_address 2020-01-10 13:57:52 -05:00
Jack Robison 08f6520557 bare excepts 2020-01-10 12:27:56 -05:00
Jack Robison 38b108752e batched blockchain.address.subscribe 2020-01-09 23:40:04 -05:00
Jack Robison 0ee7870bdf defaultdict 2020-01-09 23:06:29 -05:00
Jack Robison 37d46ecdb2 fix looping over same things in _transaction_io 2020-01-09 23:06:29 -05:00
Oleg Silkin 2a7911c7d6 rename jsonrpc_comment_edit -> jsonrpc_comment_update 2020-01-09 19:28:45 -05:00
Oleg Silkin fc2d9b4fd2 restart travis build 2020-01-09 19:28:45 -05:00
Oleg Silkin 832020fa81 Makes claim_id and parent_id mutually exclusive for comment create 2020-01-09 19:28:45 -05:00
Oleg Silkin 78606ed4b8 Generates docs for comments 2020-01-09 19:28:45 -05:00
Oleg Silkin a04ba606e6 Adds test for comment_edit 2020-01-09 19:28:45 -05:00
Oleg Silkin ebf2e7ee70 Better validation logic; 2020-01-09 19:28:45 -05:00
Oleg Silkin b31881f424 Removes possibility of no channel being passed in & propogates errors 2020-01-09 19:28:45 -05:00
Oleg Silkin c0a6f6fd08 Adds validator methods 2020-01-09 19:28:45 -05:00
Oleg Silkin 7e1b1ca730 Adds comment_edit command 2020-01-09 19:28:45 -05:00
Jack Robison 163d176f0a Merge pull request #2710 from lbryio/fix-version-and-stop
Fix `version` and `stop` rpcs
2020-01-08 17:42:10 -05:00
Jack Robison 0f7eab5100 pylint 2020-01-08 12:44:22 -05:00
Jack Robison cbf78d474a fix version and stop rpcs 2020-01-08 12:44:22 -05:00
Victor Shyba 3615e0de34 cleanup leftover comment from review 2020-01-07 19:57:34 -05:00
Victor Shyba e73c081d36 remove bandwidth_limit env option 2020-01-07 19:57:34 -05:00
Victor Shyba 663bf6e3af remove bandwidth_limit from docker file 2020-01-07 19:57:34 -05:00
Victor Shyba 60194cbafd remove bw_limit on server 2020-01-07 19:57:34 -05:00
Lex Berezhny 7d2eb5faf7 v0.52.0 2020-01-06 12:21:49 -05:00
Lex Berezhny 5969f3d213 revert version 2020-01-06 12:20:05 -05:00
Lex Berezhny 6ddde88389 update release script with new directory structure 2020-01-06 12:18:26 -05:00
Lex Berezhny e9992ed6b4 fix test 2020-01-06 12:17:09 -05:00
Lex Berezhny 4db823a3be set version to 0.52.0 2020-01-06 12:17:09 -05:00
Lex Berezhny 78732513c8 set wallet server min/max 0.52.0-0.99.0 2020-01-06 12:17:09 -05:00
Jack Robison 79557bb878 Merge pull request #2704 from lbryio/disable-cryptonator
disable cryptonator feed
2020-01-03 16:18:05 -05:00
Jack Robison 4094b02ae9 update test 2020-01-03 16:00:00 -05:00
Jack Robison 9cc7c118a8 disable cryptonator feed 2020-01-03 15:28:29 -05:00
Lex Berezhny d3dad51c76 fix import error in cli.py 2020-01-03 03:08:15 -05:00
Lex Berezhny 5bf35de955 fix blockchain integration tests 2020-01-03 03:08:15 -05:00
Lex Berezhny f8928c654b do not reset callable_methods on Daemon class 2020-01-03 03:08:15 -05:00
Lex Berezhny b3903f4ffd fix import in unit test 2020-01-03 03:08:15 -05:00
Lex Berezhny 46c3f76edc run all integration tests on travis until we drop it 2020-01-03 03:08:15 -05:00
Victor Shyba 3488408b7a add jobs=8 back to pylint 2020-01-03 03:08:15 -05:00
Victor Shyba b7b164b84c fix comment alignment 2020-01-03 03:08:15 -05:00
Victor Shyba 7c353b7d76 too many \ 2020-01-03 03:08:15 -05:00
Lex Berezhny 3d9841b61a fix unit wallet tests 2020-01-03 03:08:15 -05:00
Lex Berezhny 87e2d2c7b0 re-enable pylint on CI 2020-01-03 03:08:15 -05:00
Lex Berezhny ff59619af4 more pylint 2020-01-03 03:08:15 -05:00
Lex Berezhny b79f2b86b5 renamed Daemon.py to daemon.py per pylint 2020-01-03 03:08:15 -05:00
Lex Berezhny 386fd7a459 updated imports after renaming component files 2020-01-03 03:08:15 -05:00
Lex Berezhny d4f41901ef lower cased component file names per pylint 2020-01-03 03:08:15 -05:00
Lex Berezhny f170da3e78 pylint fixes in lbry/extras/daemon 2020-01-03 03:08:15 -05:00
Lex Berezhny 5dc15be98a pylint lbry/extras/daemon/exchange_rate_manager.py 2020-01-03 03:08:15 -05:00
Lex Berezhny 88c7cfc745 pylint in progress in lbry/extras/daemon 2020-01-03 03:08:15 -05:00
Victor Shyba 867478697d lint blob/* 2020-01-03 03:08:15 -05:00
Lex Berezhny d27e8cf73a pylint fix in lbry/error/generate.py 2020-01-03 03:08:15 -05:00
Victor Shyba 4490caa4f7 lint storage 2020-01-03 03:08:15 -05:00
Lex Berezhny 1b25f2c531 pylint fix in lbry/utils.py 2020-01-03 03:08:15 -05:00
Lex Berezhny 9d911d1fa0 convince pylint about descriptor return type in lbry/conf.py 2020-01-03 03:08:15 -05:00
Victor Shyba 494feb9f6d lint: lbry/stream/* 2020-01-03 03:08:15 -05:00
Lex Berezhny 2a04943a67 add l variable for lists to allowed good-names in pylint 2020-01-03 03:08:15 -05:00
Lex Berezhny 731b29ce91 progress on pylint in lbry/conf.py 2020-01-03 03:08:15 -05:00
Victor Shyba 28fbb70858 blob_exchange lint 2020-01-03 03:08:15 -05:00
Victor Shyba 44f402c64e delete cryptoutils 2020-01-03 03:08:15 -05:00
Victor Shyba c7f391ca44 lint: finish dht parts 2020-01-03 03:08:15 -05:00
Lex Berezhny 5ed9c5e168 ignore pylint for winpaths.py 2020-01-03 03:08:15 -05:00
Lex Berezhny 51b1a86d19 pylint for lbry/connection_manager.py 2020-01-03 03:08:15 -05:00
Victor Shyba 20c46677d0 lint dht: datastore protocol 2020-01-03 03:08:15 -05:00
Lex Berezhny efb5f232f7 fixed pylint for lbry/utils.py 2020-01-03 03:08:15 -05:00
Lex Berezhny 9edf9561b2 pylint 2020-01-03 03:08:15 -05:00
Lex Berezhny 5d4eb018ee pylint for lbry/testcase.py 2020-01-03 03:08:15 -05:00
Victor Shyba 10fbce056b dht constants -> CONSTANTS (linting) 2020-01-03 03:08:15 -05:00
Lex Berezhny cbc6d6a572 pylint passes for lbry/wallet 2020-01-03 03:08:15 -05:00
Lex Berezhny fb1af9e3d2 update imports and more merging 2020-01-03 03:08:15 -05:00
Lex Berezhny c9e410a6f4 merged torba base classes with lbry sub-classes 2020-01-03 03:08:15 -05:00
Lex Berezhny c8d72b59c0 fix readme urls 2020-01-01 16:02:16 -05:00
Lex Berezhny cd71d4a2f7 remove double install of coverage package 2020-01-01 16:02:11 -05:00
Lex Berezhny ed0cc59e66 fix cyclic import error 2020-01-01 15:57:56 -05:00
Lex Berezhny d4f1b48f52 renamed integration tests to be clearer in gitlab UI 2020-01-01 15:57:56 -05:00
Lex Berezhny 08bec02170 added __init__ 2020-01-01 15:57:56 -05:00
Lex Berezhny c9f27b83e1 split integration tests into three jobs 2020-01-01 15:57:56 -05:00
Lex Berezhny 8327585b3b fix set_build.py and other things 2020-01-01 15:57:56 -05:00
Lex Berezhny 727815d6dd updating scripts after moving lbry up one level 2020-01-01 15:57:56 -05:00
Lex Berezhny 2968f74c6c moved lbry up one level 2020-01-01 15:57:56 -05:00
Lex Berezhny 431499f43f do not install torba 2020-01-01 15:57:56 -05:00
Lex Berezhny d809283d23 fix integrartion test 2020-01-01 15:57:56 -05:00
Lex Berezhny b49275fe6b added support for --no-logging argument to avoid leaking loggers in tests 2020-01-01 15:57:56 -05:00
Lex Berezhny 87c7ce588e disabled cryptonator.com in tests, removed references to torba 2020-01-01 15:57:56 -05:00
Lex Berezhny 26d0a7c742 fix generate_json_api 2020-01-01 15:57:56 -05:00
Lex Berezhny b01887c7ec fix two unit tests 2020-01-01 15:57:56 -05:00
Lex Berezhny 745031c020 orchstr8 works again, for now... 2020-01-01 15:57:56 -05:00
Lex Berezhny 3ccfc09e01 conslidated testcase 2020-01-01 15:57:56 -05:00
Lex Berezhny e8750275c9 skip code quality checks, will fix later 2020-01-01 15:57:56 -05:00
Lex Berezhny f0d7ea4cc6 updated files and scripts post torba merge 2020-01-01 15:57:56 -05:00
Lex Berezhny ea5322af82 removing bitcoin support 2020-01-01 15:57:56 -05:00
Lex Berezhny 0b23f68fb2 merged torba into lbry 2020-01-01 15:57:56 -05:00
Lex Berezhny 1c00129f76 v0.51.2 2019-12-30 16:34:52 -05:00
Lex Berezhny 454700af05 support json in setting_set 2019-12-30 16:31:14 -05:00
Jack Robison d9b482d90a Merge pull request #2699 from lbryio/faster-account-startup
don't log account details on startup if there are more than 10
2019-12-30 16:26:36 -05:00
Jack Robison 09c2c97069 only log account details on startup if there are not very many 2019-12-30 16:22:41 -05:00
Lex Berezhny 88faf0ce78 v0.51.1 2019-12-30 15:11:29 -05:00
Jack Robison aef83eec31 Merge pull request #2695 from lbryio/fix-12-to-13-migrator
Fix 12 to 13 migrator
2019-12-28 15:57:41 -05:00
Jack Robison 5bd222c266 less verbose log 2019-12-28 13:17:19 -05:00
Jack Robison d3e7f789b2 uncaught exception 2019-12-28 13:17:19 -05:00
Jack Robison 87c6e292f1 fix wallet server SessionManager._get_info 2019-12-28 13:17:19 -05:00
Jack Robison b597ad0b3f fix unique constraint failure in migrate12to13 2019-12-28 13:17:19 -05:00
Alex Grintsvayg 40dd99dea3 changelog note 2019-12-28 12:39:46 -05:00
Lex Berezhny 6ea2686219 v0.51.0 2019-12-23 12:39:48 -05:00
Alex Grintsvayg 6bca90c4f7 allow wallet snapshots using different compression algos 2019-12-21 19:18:20 -05:00
Alex Grintsvayg 20fa7bd852 better server range for boris 2019-12-21 13:48:07 -05:00
Alex Grintsvayg 6b80119eff drop travis badge 2019-12-21 13:29:09 -05:00
Alex Grintsvayg 1fc5469eb1 add gitlab badge 2019-12-21 13:26:38 -05:00
Alex Grintsvayg bf2769a0c2 disable travis docker builds. docker builds trigger from docker hub 2019-12-21 13:06:03 -05:00
Victor Shyba 325b601bea enable WAL 2019-12-21 07:49:21 -05:00
Lex Berezhny 126642912e lint 2019-12-20 16:52:56 -05:00
Lex Berezhny 57ee16d565 change returned error data structure to be JSONRPC standard compliant 2019-12-20 16:52:56 -05:00
Alex Grintsvayg 9f0fba063d ensure asset exists before upload 2019-12-20 13:01:51 -05:00
Victor Shyba 757fb51415 make block_expected public 2019-12-20 12:30:16 -05:00
Victor Shyba b1c5655138 __height -> _block_expected 2019-12-20 12:30:16 -05:00
Alex Grintsvayg 4d97dcaa93 remove extraneous apostrophe 2019-12-20 10:26:41 -05:00
Jack Robison 96db1b4685 v0.50.1 2019-12-19 22:09:19 -05:00
Jack Robison 9e21c52d04 Merge pull request #2685 from lbryio/fast_broadcast_notify
Update balance after a broadcast transaction is accepted to mempool without a delay, fix bug in `status` header progress
2019-12-19 22:07:16 -05:00
Victor Shyba 5fa801f9f2 broadcast_or_release uses None as timeout as opposed to tests 2019-12-19 23:42:31 -03:00
Victor Shyba 0301768b79 lint on tests 2019-12-19 22:58:36 -03:00
Victor Shyba 4c6dedfa4f fix last test 2019-12-19 22:34:10 -03:00
Victor Shyba 6652d55455 wip 2019-12-19 21:46:14 -03:00
Victor Shyba 3e03dd3e80 apply timeout and check for expected height on wait 2019-12-19 20:20:42 -03:00
Victor Shyba 84b8a22423 comment the broadcast interruption case 2019-12-19 16:20:12 -03:00
Victor Shyba ada03e12fc proper check for restrictions on gettx 2019-12-19 15:13:13 -03:00
Victor Shyba d66c801350 wake up mempool on broadcast 2019-12-19 15:13:13 -03:00
Victor Shyba 91846939f6 organize logic for when its downloading 2019-12-18 19:09:33 -03:00
Jack Robison c85127d76b fix division by zero error 2019-12-18 10:44:16 -05:00
Lex Berezhny 166bf65e88 torba test_database fix 2019-12-17 23:48:07 -05:00
Lex Berezhny f68bdc406f fix bug when two updates for the same claim happened in the same block 2019-12-17 23:48:07 -05:00
Lex Berezhny 8390482d6c aws does not use memoryview 2019-12-17 23:48:07 -05:00
Alex Grintsvayg 783f454f8e dont include dist dir in windows zip 2019-12-16 20:32:42 -05:00
Alex Grintsvayg 86a67bd245 autopublish release and notify in slack 2019-12-16 16:40:56 -05:00
Alex Grintsvayg 099b0b65c0 run torba unit tests the way lex runs them 2019-12-16 12:59:15 -05:00
Lex Berezhny 505652efa0 v0.50.0 2019-12-16 11:33:13 -05:00
Victor Shyba a20088330f use buffer write return as written 2019-12-16 11:31:28 -05:00
Jack Robison a20e2504bb disable timing bug in test_ping_queue_discover 2019-12-16 10:10:58 -05:00
Lex Berezhny b07bc150e1 do not registery fake component in real component registry 2019-12-15 02:39:17 -05:00
Lex Berezhny 46c6588aa1 fixup fake exchange rate manager for integration tests 2019-12-15 02:39:17 -05:00
Lex Berezhny e36c672c9a pylint 2019-12-15 02:39:17 -05:00
Lex Berezhny c044e1ea8c further refactor and simplification 2019-12-15 02:39:17 -05:00
Lex Berezhny 25b6c1b6ca renamed test_ExchangeRateManager to test_exchange_rate_manager 2019-12-15 02:39:17 -05:00
Lex Berezhny c25d72d911 minor error class and type checking fixes 2019-12-15 02:39:17 -05:00
Miroslav Kovar 73613d1583 Refactor exchange rate manager 2019-12-15 02:39:17 -05:00
Lex Berezhny 1e6542d12d add --blocklist_channel_ids to claim_search command 2019-12-14 19:19:33 -05:00
Lex Berezhny 3cffaa43f7 remove invalid claim_id from default channel filter list 2019-12-14 19:19:33 -05:00
Lex Berezhny 08792e794f a support create and abandon in same block no longer inserts support 2019-12-14 18:19:06 -05:00
Lex Berezhny 7fa26d59da fixes for various edge cases where claim creation, updating and abandoning is occuring in the same block 2019-12-14 18:19:06 -05:00
Lex Berezhny 74469829e5 pylint 2019-12-14 15:47:28 -05:00
Lex Berezhny ba414742a6 improved the data structure returned from RPC for errors by adding a error name and other metadata 2019-12-14 15:47:28 -05:00
Lex Berezhny f6fcfb6dde simplified and only log potentially transactional functions 2019-12-13 18:44:32 -05:00
Oleg Silkin c5b90c0144 Removes unused debug flag 2019-12-13 18:44:32 -05:00
Oleg Silkin f669881849 Adds function, param logging to all lbrynet handling 2019-12-13 18:44:32 -05:00
Alex Grintsvayg d0b70ea6df upload assets from gitlab CI to github on a tagged release
also disables travis asset uploading
2019-12-13 13:35:40 -05:00
Lex Berezhny 51af43d492 move none_check after fee update since that can be None 2019-12-13 13:23:23 -05:00
Lex Berezhny 8a49ad4586 catch invalid None value for all cases when creating/updating claims 2019-12-13 13:23:23 -05:00
gpjacobs 2ae068c2d2 Improving error handling for null values 2019-12-13 13:23:23 -05:00
Alex Grintsvayg 5e07e8dbbd copy some of lex's changes to boris 2019-12-13 10:12:47 -05:00
Lex Berezhny 8d2c68505e remove VERBOSITY from test 2019-12-12 13:25:27 -05:00
Lex Berezhny 44f1e7c401 test all possible P2SH and BECH32 payment combinations on wallet server and at least P2SH and BECH32 to P2PKH receipt on client address 2019-12-12 13:25:27 -05:00
Lex Berezhny fd2945469e pylint 2019-12-12 13:25:27 -05:00
Lex Berezhny b0f0827cff client side segwit support 2019-12-12 13:25:27 -05:00
Lex Berezhny c1e2c415c4 TaskGroup cannot work without even loop 2019-12-11 22:48:39 -05:00
Victor Shyba a258b96602 simplify test 2019-12-11 22:48:39 -05:00
Victor Shyba 82ef76b1cb set regardless, there is no need for the check 2019-12-11 22:48:39 -05:00
Victor Shyba aaa9982932 missing test file 2019-12-11 22:48:39 -05:00
Victor Shyba 1cdb4bc9e7 task group: set done if empty 2019-12-11 22:48:39 -05:00
Alex Grintsvayg fddb0c9e66 notify us when boris is killed with signals 2019-12-11 14:02:39 -05:00
Lex Berezhny 5c78d2b6d2 install wget in Dockerfile 2019-12-11 13:08:02 -05:00
Victor Shyba a4113cbb52 add apsw to pylint conf 2019-12-11 12:38:00 -05:00
Victor Shyba afa3bda3c8 update tests 2019-12-11 12:06:41 -05:00
Victor Shyba 61d7edd15f change api name to wallet reconnect 2019-12-11 12:06:41 -05:00
Victor Shyba c2106de7e1 refactor + tests 2019-12-11 12:06:41 -05:00
Victor Shyba 0bf1be6198 add wallet restart API 2019-12-11 12:06:41 -05:00
Lex Berezhny af7c20e440 downgrade multidict to prevent segfault 2019-12-11 09:44:37 -05:00
Lex Berezhny 17771a2b89 added --initial-headers argument to lbrynet start command 2019-12-11 09:44:37 -05:00
Lex Berezhny 8840097fe2 test that creator can spend purchases received 2019-12-09 22:32:14 -05:00
Lex Berezhny 7d333efd45 utxo list includes purchases of your claims 2019-12-09 22:32:14 -05:00
Lex Berezhny ec65815c57 add parsing for segwit outpoint 2019-12-09 20:59:31 -05:00
Lex Berezhny 640b0c71e6 v0.49.0 2019-12-09 13:05:08 -05:00
Victor Shyba 4208876ef3 add header sync to update tasks so it can be cancelled 2019-12-09 13:04:23 -05:00
Victor Shyba bff9117360 progress from 0 2019-12-09 13:04:23 -05:00
Victor Shyba 6b3c4c70d2 dont timeout if data being transferred 2019-12-09 13:04:23 -05:00
Lex Berezhny 0292fd8d91 fix lint error 2019-12-08 22:40:06 -05:00
Lex Berezhny 94a41270d0 add InvalidPasswordError handling to more places 2019-12-08 22:40:06 -05:00
Lex Berezhny 75d78bfa53 added InvalidPasswordError code when password is invalid 2019-12-08 22:40:06 -05:00
Victor Shyba 88263db831 moar variables limit 2019-12-08 17:00:41 -05:00
Lex Berezhny 3ef96404ee use make install in Dockerfile 2019-12-08 17:00:41 -05:00
Lex Berezhny 13f9370f8c fix wallet server db writer test 2019-12-08 17:00:41 -05:00
Lex Berezhny 1c349270bf switch to apsw 2019-12-08 17:00:41 -05:00
Lex Berezhny c28f0f6286 added apsw installation to tox 2019-12-08 17:00:41 -05:00
Lex Berezhny b20939d211 added apsw installation 2019-12-08 17:00:41 -05:00
Lex Berezhny b76c45fc2b drop jonathonf/backports sqlite3 PPA from .travis.yml since the package was removed 2019-12-08 17:00:41 -05:00
Lex Berezhny 1bbfccd082 refactored error codes generate script and error hierarchy 2019-12-06 10:45:23 -05:00
Lex Berezhny cee7e06832 add configurable log level to exceptions, join message and description columns in error markdown table 2019-12-06 10:45:23 -05:00
Jack Robison 03dff4b382 fix build 2019-12-05 20:56:57 -05:00
Jack Robison f546b20a01 v0.48.2 2019-12-04 13:10:40 -05:00
Jack Robison 0e8d9a264f Merge pull request #2601 from mirgee/fix-2490
Persist and restore routing table
2019-12-03 15:44:01 -05:00
Jack Robison 66a4c98bee update test 2019-12-03 13:11:34 -05:00
Miroslav Kovar 36101db500 Wait for routing table to clear with timeout 2019-12-03 13:06:50 -05:00
Miroslav Kovar 6bff298d1e Add migrator for the new peer table 2019-12-03 13:06:50 -05:00
Jack Robison c832f8ffbb fix mock_network_loop param 2019-12-03 13:06:50 -05:00
Jack Robison d7fe46dbde fix drop/reconnect in test_losing_connection 2019-12-03 13:06:50 -05:00
Miroslav Kovar 5951186463 Minor changes
Fix typos
2019-12-03 13:06:50 -05:00
Miroslav Kovar 7191042bb8 Add new test, remove old test, change error type 2019-12-03 13:06:50 -05:00
Miroslav Kovar a80fbcc252 Catch resolve timeouts 2019-12-03 13:06:50 -05:00
Miroslav Kovar c321758afd Rename, fix tests / deduplicate set_joined, resolve conditionally 2019-12-03 13:06:50 -05:00
Miroslav Kovar ca8f59a643 Persist and restore routing table 2019-12-03 13:06:50 -05:00
Jack Robison 8110882617 Merge pull request #2650 from lbryio/fix-blob-request-loop
Fix blob request loop
2019-12-03 12:58:27 -05:00
Jack Robison a283b66541 add timeout to file_save attempt starting
-remove test_unban_recovers_stream
2019-12-03 11:56:59 -05:00
Jack Robison 569de37e16 fix sendfile exceptions 2019-12-03 11:56:55 -05:00
Jack Robison 669f3394c7 fix requesting same blob over and over if only peer(s) say they don't have it 2019-12-03 11:56:52 -05:00
Alex Grin 87ec2c7378 v0.48.1 2019-12-02 12:39:41 -05:00
Jack Robison ea7a31cad1 Merge pull request #2649 from lbryio/invalid-ips
Fix handling reserved ips
2019-12-01 18:06:32 -05:00
Jack Robison c894ade25e fix integration tests 2019-12-01 17:28:51 -05:00
Jack Robison a3fe127a92 fix and test reserved ips 2019-12-01 17:04:03 -05:00
Jack Robison 7f526be879 Merge pull request #2648 from lbryio/fix-blob-udp-ping
Fix UDP ping port in blob peer accumulator
2019-12-01 17:00:12 -05:00
Jack Robison 71ae174e29 fix predicting udp port in _accumulate_peers_for_value
-add comments
-rename functions to be more descriptive of what they do
2019-12-01 16:21:15 -05:00
Jack Robison 2277d134cc Merge pull request #2647 from lbryio/drain-iterative-findvalue
Request all pages of peers for a blob
2019-12-01 15:52:28 -05:00
Jack Robison 880aa265f1 drain all pages of peers returned by find_value in IterativeValueFinder 2019-12-01 14:53:39 -05:00
Alex Grintsvayg ea639e64a4 add docker-compose file for wallet server 2019-11-27 10:18:39 -05:00
Lex Berezhny b60fdee5e2 upgrade SDK to use lbrycrd v0.17.3.2 2019-11-26 21:31:31 -05:00
Alex Grintsvayg 44f07fb81d move build-related files into top-level dir 2019-11-26 18:28:58 -05:00
Alex Grintsvayg 60a85fadd3 no more codecov 2019-11-26 18:28:58 -05:00
Alex Grintsvayg 68367125e1 gitlab builds on windows
what's left:
  - github build progress notifications (need to upgrade to Gitlab Silver for this)
  - github releases
  - uploading coverage results to codecov.io
  - what directories to cache between builds
2019-11-26 18:28:58 -05:00
Alex Grintsvayg 6112f60681 gitlab ci setup
what works:
- tests
- linux and mac builds
- uploading builds to s3

what's left to do:
- uploading coverage results to codecov.io
- windows build
- github releases
- what directories to cache between builds
2019-11-26 18:28:58 -05:00
Alex Grintsvayg ecc74e2ae5 if user has no Downloads dir on windows, use cwd 2019-11-26 18:28:58 -05:00
jessop c2dae43965 response 2019-11-26 14:45:41 -05:00
jessop 6e3a25d255 status: wallet returns all servers rather than just available 2019-11-26 14:45:41 -05:00
jessop 4e13cf1d85 adds wallet server counts on startup 2019-11-26 14:45:41 -05:00
Lex Berezhny 0fee1897eb v0.48.0 2019-11-25 20:39:36 -05:00
Victor Shyba 9dd51467fe move lbry specific version to its place 2019-11-25 20:21:13 -05:00
Victor Shyba cec6dca72f bump protocol version 2019-11-25 20:21:13 -05:00
Victor Shyba 58df2b9bcf ignore cancelled error 2019-11-25 16:07:03 -05:00
Victor Shyba cc6bdd6295 sync fts only once 2019-11-25 16:07:03 -05:00
Victor Shyba 7a044f9f78 opening dbs can raise too 2019-11-25 16:07:03 -05:00
Victor Shyba e77acde005 wip 2019-11-25 16:07:03 -05:00
Victor Shyba ece5082096 enable callback tracebacks 2019-11-25 16:07:03 -05:00
Alex Grintsvayg 0723026a56 drop codecov 2019-11-25 12:52:30 -05:00
Lex Berezhny 0b7346f4a1 droped OldWalletServerTransaction class in wallet server unit tests 2019-11-25 10:22:34 -05:00
Lex Berezhny a357db1ff6 use correct segwit deserializer 2019-11-25 10:22:34 -05:00
Jack Robison a2287f8d0a Merge pull request #2637 from lbryio/fix-unhandled-download-errors
don't log tracebacks for expected download errors
2019-11-25 10:11:37 -05:00
Jack Robison 076decd6e9 don't log tracebacks for expected download errors 2019-11-25 09:47:42 -05:00
jessop d40461de94 review changes 2019-11-22 22:28:55 -05:00
jessop 4b1492be0d return value and more tests 2019-11-22 22:28:55 -05:00
jessop 1676246743 cleanup 2019-11-22 22:28:55 -05:00
jessop f7c9f6d142 adds settings_clear method and integration test 2019-11-22 22:28:55 -05:00
Alex Grintsvayg 6022339091 Revert "dont show loop blocking warnings when running tests in CI"
This reverts commit 25d97afa60.
2019-11-22 20:27:10 -05:00
Alex Grintsvayg 25d97afa60 dont show loop blocking warnings when running tests in CI 2019-11-22 11:55:04 -05:00
Alex Grintsvayg df635b963c enable passing args to tests 2019-11-21 15:31:49 -05:00
Lex Berezhny 6d0a054170 regenerate latest docs with claim_id argument to stream_repost 2019-11-20 10:56:42 -05:00
Lex Berezhny 177d3af77f regenerate latest docs 2019-11-20 10:39:17 -05:00
Lex Berezhny 35f71c619b fix unit tests 2019-11-19 22:26:30 -05:00
Lex Berezhny 31a80ccce4 minor fixups 2019-11-19 22:26:30 -05:00
Lex Berezhny 3194cec8df added missing files 2019-11-19 22:26:30 -05:00
Lex Berezhny 57fd3c5801 updated code base to use generated errors 2019-11-19 22:26:30 -05:00
Lex Berezhny 86617f1bda added names to errors and script to generate error classes from README 2019-11-19 22:26:30 -05:00
Lex Berezhny 35f36c598c moved error.py to be a module 2019-11-19 22:26:30 -05:00
Lex Berezhny 36bd7816fe add handling for invalid claims in calculate_reposts 2019-11-18 17:07:46 -05:00
Lex Berezhny fd632392d4 full reposted_claim data returned for claim_search and resolve 2019-11-18 17:07:46 -05:00
Victor Shyba 423b48866f apply refactor suggested on review by removing .out calls 2019-11-18 17:07:46 -05:00
Victor Shyba 745bc24343 add a test for normal repost mixed with blocking repost, use inner queries 2019-11-18 17:07:46 -05:00
Victor Shyba 04a823c7d0 FILTERING_CHANNELS_IDS on claim search 2019-11-18 17:07:46 -05:00
Victor Shyba 56af4c2fcb minor fixes from rebase 2019-11-18 17:07:46 -05:00
Victor Shyba 77daf6812a add config option for filtering channels 2019-11-18 17:07:46 -05:00
Victor Shyba c7e964ec42 add reposted_claim_hash column, writer and reader w/ tests 2019-11-18 17:07:46 -05:00
Victor Shyba 694e2c2a4f improve test cases for reposts, add a new one for reverse search 2019-11-18 17:07:46 -05:00
Victor Shyba df6537fae8 add API for reposting 2019-11-18 17:07:46 -05:00
Lex Berezhny 71f6542571 v0.47.0 2019-11-18 10:57:58 -05:00
Victor Shyba 5a6294c08e test errors arent trapped 2019-11-18 10:57:24 -05:00
Victor Shyba f0390786d6 avoid trapping errors on client 2019-11-18 10:57:24 -05:00
Jack Robison 719d18c670 Merge pull request #2620 from lbryio/torrent-db-support
Add torrent support to SQLiteStorage
2019-11-15 18:34:00 -05:00
Jack Robison a116dcd3bb add SQLiteStorage.delete_torrent 2019-11-15 16:53:19 -05:00
Jack Robison e2dd3dcf88 get_claims_from_torrent_info_hashes 2019-11-15 16:53:19 -05:00
Jack Robison 822f0c3cb2 add migrate12to13 script 2019-11-15 16:53:14 -05:00
Jack Robison 63fb39016b add torrent db tables
-decouple file table from stream table
2019-11-15 15:25:33 -05:00
Jack Robison e79f8d58ca expose bt_infohash in Claim 2019-11-15 15:25:33 -05:00
Lex Berezhny a469dfb583 stop signing support tx too many times 2019-11-15 14:16:54 -05:00
Victor Shyba 6cbc545f84 fix .connect injected method 2019-11-15 13:59:28 -05:00
Victor Shyba ea1e24d8f9 refactor from review 2019-11-15 13:59:28 -05:00
Victor Shyba 6fcb4d93c9 header component now shows header synchronization progress 2019-11-15 13:59:28 -05:00
Victor Shyba 36031c9d0f fix test, off by 1 2019-11-15 13:59:28 -05:00
Victor Shyba a70f5e2440 update block status during download 2019-11-15 13:59:28 -05:00
Victor Shyba 6c42aab567 stop spurious warning from wrong check 2019-11-15 13:59:28 -05:00
Victor Shyba 27631a1746 test initial header sync separately 2019-11-15 13:59:28 -05:00
Victor Shyba 320f5cebc7 cleanup 2019-11-15 13:59:28 -05:00
Victor Shyba ea2a583803 wip 2019-11-15 13:59:28 -05:00
Victor Shyba 9965801258 base64 headers 2019-11-15 13:59:28 -05:00
Victor Shyba 24ed0521c7 checkpoints 2019-11-15 13:59:28 -05:00
Victor Shyba d8fed79810 remove headers component 2019-11-15 13:59:28 -05:00
Victor Shyba c30db15efa faster header download 2019-11-15 13:59:28 -05:00
Jack Robison 817869b915 Merge pull request #2623 from lbryio/aioupnp-0.0.16
bump aioupnp to 0.0.16, fixes UPnP support for TP-Link routers
2019-11-15 11:50:59 -05:00
Jack Robison 3038dd9440 bump aioupnp -> 0.0.16 2019-11-15 11:24:49 -05:00
Lex Berezhny 276b2c737a pylint 2019-11-14 18:53:20 -05:00
Lex Berezhny 62543b94c9 minor cleanup of collections implementation 2019-11-14 18:53:20 -05:00
jessop 4d607ed276 review response 2019-11-14 18:53:20 -05:00
jessop 26e2865e35 cleanup 2019-11-14 18:53:20 -05:00
jessop f7c94c9eb5 collection resolve endpoint 2019-11-14 18:53:20 -05:00
jessop 25a91f89fd tests 2019-11-14 18:53:20 -05:00
jessop 246f055145 enables collections 2019-11-14 18:53:20 -05:00
Lex Berezhny 8c3bf0d4cb try ubuntu bionic on travis 2019-11-14 17:19:10 -05:00
Lex Berezhny e49f24e95f added more tests to make sure stream update and abandon update the full text search indexes 2019-11-14 17:19:10 -05:00
Lex Berezhny 48a18ff771 full text search in wallet server 2019-11-14 17:19:10 -05:00
Alex Grintsvayg 37161057c6 plz explain errors 2019-11-14 09:52:30 -05:00
Alex Grintsvayg 68d1157495 catch the right error, boris 2019-11-13 17:58:47 -05:00
Alex Grintsvayg 19c3b2f4b6 add a latest-release tag to wallet server docker builds 2019-11-13 12:08:04 -05:00
Lex Berezhny 7051c332a1 v0.46.1 2019-11-12 11:26:24 -05:00
Lex Berezhny 0f2a0b7c97 added witness transaction parsing to torba parser 2019-11-12 11:24:42 -05:00
Lex Berezhny bb6ef42d0c added support for lbrycrd v0.17.3.1 2019-11-12 11:24:42 -05:00
Jack Robison 3c591d4968 update time_to_first_bytes.py 2019-11-11 20:43:52 -05:00
Lex Berezhny cf356a8e35 v0.46.0 2019-11-11 10:49:44 -05:00
Victor Shyba 3a0ce58cda send back correct server version 2019-11-11 10:14:00 -05:00
Lex Berezhny 2d7038dc18 fixed wallet balance for multiple accounts 2019-11-05 10:14:03 -05:00
Lex Berezhny 25a0e67841 added caching for account/wallet balance and removed --reserved_subtotals argument instead always returning the subtotals 2019-11-05 10:14:03 -05:00
Alex Grintsvayg be64209292 less verbose snapshot download 2019-11-04 16:51:57 -05:00
Lex Berezhny 0006cb1c27 v0.45.0 2019-11-04 10:30:57 -05:00
Lex Berezhny 836d38ed2a added claim_trending_group_mixed_idx index to wallet server db 2019-11-04 10:28:22 -05:00
Thomas Zarebczan ec71017785 update trending period
This gets us back to ~7 days, which was originally requested as part of this change.
2019-11-04 10:28:22 -05:00
Jack Robison faf9449107 Merge pull request #2596 from lbryio/dont-log-incompatible-peer-error
don't log tracebacks for old peers failing to handle findValue requests
2019-11-01 13:13:38 -04:00
Jack Robison f93bbd6bc3 improve reflector logging 2019-11-01 11:32:36 -04:00
Jack Robison ca12d655ea don't log tracebacks for old peers failing to handle findValue requests during blob announcement 2019-11-01 11:30:44 -04:00
Jack Robison 72e486791b Merge pull request #2552 from lbryio/dht-extensions
Add optional extensions to DHT datagram format
2019-11-01 11:27:44 -04:00
Jack Robison 874c28bd88 add optional_fields to KademliaDatagramBase
-update KademliaDatagramBase.bencode and decode_datagram
2019-11-01 10:47:45 -04:00
Jack Robison aa7c0a3544 test unused fields are backwards compatible 2019-11-01 10:45:12 -04:00
Lex Berezhny 3624a3b450 added --uri to purchase_list and file_list now includes purchase_receipt field 2019-10-30 23:00:45 -04:00
Lex Berezhny c4c50699cc fix transaction_list for purchases correctly showing balance_delta 2019-10-30 23:00:45 -04:00
Lex Berezhny 6a3d760cfe fix for integration test 2019-10-30 23:00:45 -04:00
Lex Berezhny 8ceaf66a81 pylint fixes 2019-10-30 23:00:45 -04:00
Lex Berezhny fdd2562f32 added purchase_list and purchase_create 2019-10-30 23:00:45 -04:00
Lex Berezhny 28457021f8 figure out why time to first byte duration went up 2019-10-30 23:00:45 -04:00
Lex Berezhny 459716bd6e initial RETURN_OP based purchase metadata 2019-10-30 23:00:45 -04:00
Lex Berezhny 41e0e6762c added wallet management tests and address unsubscribe feature on wallet server 2019-10-30 17:59:12 -04:00
Lex Berezhny a5ee60c1c9 subscribe accounts on wallet_add 2019-10-30 17:59:12 -04:00
Victor Shyba 1391c1d607 remove bandwidth and subscriptions limits from wallet server 2019-10-29 19:51:26 -04:00
Lex Berezhny 8e5d47061f check txo.has_address before calling txo.get_address() 2019-10-29 11:53:34 -04:00
Lex Berezhny 28ea43c89e v0.44.1 2019-10-28 16:00:17 -04:00
Jack Robison 59adcfe1a1 Merge pull request #2590 from lbryio/fix-upnp-external-ip-fallback
fix setting the external ip in `status` if upnp fails
2019-10-28 14:34:51 -04:00
Jack Robison cb38ba12a7 fix the external ip in status if upnp fails 2019-10-28 13:35:15 -04:00
Jack Robison dc8e146aa1 Merge pull request #2589 from lbryio/fix-db-migration
fix 11to12 migration typo
2019-10-28 13:34:41 -04:00
Jack Robison 9200f5b516 fix 11to12 migration typo 2019-10-28 12:20:15 -04:00
Lex Berezhny 5c1026c745 v0.44.0 2019-10-28 11:35:51 -04:00
Jack Robison 95e8c85aa9 Merge pull request #2588 from lbryio/bump-aioupnp
faster startup, bump aioupnp -> 0.0.15
2019-10-28 10:56:20 -04:00
Jack Robison 0d179d67a3 bump aioupnp -> 0.0.15 2019-10-28 10:19:08 -04:00
Lex Berezhny 99ce40f736 reduce output on travis and dont install python2 on mac build 2019-10-26 15:02:43 -04:00
Lex Berezhny d2315f4fd7 added ledger.on_ready to know when ledger is finished syncing 2019-10-26 15:01:17 -04:00
Lex Berezhny 6370404985 removed shutdown_runner argument to daemon.stop() 2019-10-26 15:01:17 -04:00
Lex Berezhny 1ed38fff6f removed runner.shutdown() as runner.cleanup() calls this already 2019-10-26 15:01:17 -04:00
Chris Baumler ba48713219 Add CLI unit test for keyboard interrupt 2019-10-26 15:01:17 -04:00
Lex Berezhny dfc1e6a2b9 renamed added_at to added_on, to be consistent with existing modified_on field 2019-10-26 11:57:07 -04:00
Oleg Silkin 4237331222 adds added_at field to files 2019-10-26 11:57:07 -04:00
Hugo 223bcdc261 Revert changes to generated files 2019-10-26 10:36:52 -04:00
Hugo 087124ce42 Update unit test asserts 2019-10-26 10:36:52 -04:00
Hugo c5cfd8f40e Upgrade Python syntax with pyupgrade --py37-plus 2019-10-26 10:36:52 -04:00
Hugo b70a23a10a sudo no longer needed https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration 2019-10-26 10:36:52 -04:00
Lex Berezhny ccf55bcd01 simplify line in generate_json_api.py 2019-10-26 01:18:27 -04:00
Lex Berezhny d227660d6c minor refactor and cleanup, generated api.json 2019-10-26 01:18:27 -04:00
Oleg Silkin ca6202fc19 Shortens file_list method 2019-10-26 01:18:27 -04:00
Oleg Silkin f6b5a40813 self.daemon.jsonrpc_file_list(params) -> self.file_list(params) in tests 2019-10-26 01:18:27 -04:00
Oleg Silkin 69c0cefe5e Adds pre-formatted file_list method to testcase, fixes file_list tests 2019-10-26 01:18:27 -04:00
Oleg Silkin ad8772e191 Replaces .get with bracket notation dictionary access 2019-10-26 01:18:27 -04:00
Oleg Silkin e095767bc6 Makes the page_size return default to all 2019-10-26 01:18:27 -04:00
Oleg Silkin 610f19bea1 Updates api example generation for file_list 2019-10-26 01:18:27 -04:00
Oleg Silkin 4637559c6e Enforces the default paginated output on file_list 2019-10-26 01:18:27 -04:00
Oleg Silkin 766dcff6c6 Paginates only if either --page or --page_size are provided 2019-10-26 01:18:27 -04:00
Oleg Silkin b30dc39e37 Makes page & page_size co-dependent in doc 2019-10-26 01:18:27 -04:00
Oleg Silkin a23178633a Adds tests to ensure paginated file_list works properly 2019-10-26 01:18:27 -04:00
Oleg Silkin 7c074e1cd4 paginates file_list 2019-10-26 01:18:27 -04:00
Lex Berezhny 4a8b96a6d2 removed outdated docs, see https://lbry.tech/api/sdk instead 2019-10-26 00:16:57 -04:00
Lex Berezhny 290a206012 fix api doc script 2019-10-26 00:15:19 -04:00
Lex Berezhny 7b86b3843f refactored and updates all list commands to always be paginated 2019-10-26 00:15:19 -04:00
Oleksandr Manyk 553adb2ad0 Fetch records count only when needed 2019-10-26 00:15:19 -04:00
Oleksandr Manyk cc55a78fb4 Call claim_search only once 2019-10-26 00:15:19 -04:00
Oleksandr Manyk bd1df09098 Unify paginated endpoints logic 2019-10-26 00:15:19 -04:00
Julin S c61fae05d6 Convert some format() strings in lbry and torba to fstrings 2019-10-25 20:47:39 -04:00
Lex Berezhny e3cb5bfa07 fix for stream type change integration test 2019-10-25 11:51:45 -04:00
Lex Berezhny 09f6034f2f added back prefix argument to create_upload_file fixture method 2019-10-25 07:02:21 -04:00
Lex Berezhny 63b8a76bf6 refactored tests and simplified clearing of metadata on file type change 2019-10-25 07:02:21 -04:00
Miroslav Kovar 5fe536a058 Clean up the test 2019-10-25 07:02:21 -04:00
Miroslav Kovar 342184e532 Clear metadata when updating file type 2019-10-25 07:02:21 -04:00
Lex Berezhny e5c169ddf8 minor fixup 2019-10-24 17:28:59 -04:00
Miroslav Kovar 622ba2b0e3 Revert to the original 2019-10-24 17:28:59 -04:00
Miroslav Kovar fbfe8f8d14 Remove extend, fix index var 2019-10-24 17:28:59 -04:00
Miroslav Kovar 3c96fda59b Preserve order of tags when cleaning 2019-10-24 17:28:59 -04:00
Miroslav Kovar 7cb03dde44 Normalize only when writing (setting, updating) 2019-10-24 17:28:59 -04:00
Miroslav Kovar db34f89251 Normalize tag names 2019-10-24 17:28:59 -04:00
Alex Grintsvayg 98b9972fa5 cleanup setup_logging()
added more logging tests and refactored for merging into master
2019-10-24 15:48:29 -04:00
Lex Berezhny dd5507036f fix broken tests 2019-10-23 15:29:12 -04:00
Lex Berezhny 75da209de1 fix for wallet server bug related to changing the channel of a stream 2019-10-23 15:29:12 -04:00
Lex Berezhny bb8a2bfff7 fix bug with resolve and conflicting claim_id segments 2019-10-23 11:48:00 -04:00
Lex Berezhny a7f8febed7 dont error if blank password was set previously 2019-10-23 10:45:55 -04:00
Lex Berezhny 21d291531e v0.43.4 2019-10-21 13:09:36 -04:00
Victor Shyba d133fa0d74 start virtualenv clean on Mac build 2019-10-21 13:08:19 -04:00
Victor Shyba 2653a9447b bump ecdsa version 2019-10-21 13:08:19 -04:00
Miroslav Kovar bc04903741 Return a default value 2019-10-21 12:43:40 -04:00
Miroslav Kovar 5501418727 Fix failing test 2019-10-21 12:43:40 -04:00
Miroslav Kovar 2737b3635f Revert last change 2019-10-21 12:43:40 -04:00
Miroslav Kovar e29688530d Do not close files unnecessarily 2019-10-21 12:43:40 -04:00
Miroslav Kovar 2eec26b648 Clean up 2019-10-21 12:43:40 -04:00
Miroslav Kovar d4d1d87680 Revert changes to cli, apply suggestions 2019-10-21 12:43:40 -04:00
Miroslav Kovar 869be3e361 Improve tests, update suggested file name when necessary 2019-10-21 12:43:40 -04:00
Miroslav Kovar 15376c19be Apply suggestions 2019-10-21 12:43:40 -04:00
Miroslav Kovar f92b610a03 Simplify 2019-10-21 12:43:40 -04:00
Miroslav Kovar 268f6447cb Sanitize suggested, add tests 2019-10-21 12:43:40 -04:00
Miroslav Kovar ed93c647d2 Sanitize file names before saving 2019-10-21 12:43:40 -04:00
Lex Berezhny f60db358ee v0.43.3 2019-10-18 18:10:01 -04:00
Lex Berezhny ad6bd840a9 make decrypt atomic, if it fails the state for all encrypted values in account does not change 2019-10-18 18:08:43 -04:00
Lex Berezhny 0fa9e8d9f5 if decrypted private key is empty, no longer save the encrypted empty string as private key 2019-10-18 18:08:43 -04:00
Lex Berezhny 481e265ec7 added wallet.save() in test since it is no longer saved redundantly on startup 2019-10-18 18:08:43 -04:00
Lex Berezhny 583833017b refactor save and encryption check 2019-10-18 18:08:43 -04:00
Lex Berezhny d50ebcf9eb do not save wallet if gap did not change, also dont encrypt on save if already encrypted 2019-10-18 18:08:43 -04:00
Lex Berezhny c448531551 dont save wallet on startup regardless of changes 2019-10-18 18:08:43 -04:00
Lex Berezhny 81ddf7e1d2 use an assert to check password 2019-10-18 18:08:43 -04:00
Lex Berezhny 180035296e support unlocking with blank password but throw an error when encrypting with blank password 2019-10-18 18:08:43 -04:00
Jack Robison afe24a3e65 update time_to_first_byte.py 2019-10-18 15:59:21 -04:00
Jack Robison 9058201de0 update download_blob_from_peer.py 2019-10-18 15:59:21 -04:00
Jack Robison 49a54f8a13 fix total_sent and total_received in connection status
- remove `time` field from connection status
- add `max_incoming_mbs` and `max_outgoing_mbs` to connection status
- reduce delay in connection status loop
2019-10-18 15:59:21 -04:00
Lex Berezhny 5cebc7f5cd pylint 2019-10-18 12:02:36 -04:00
Lex Berezhny 4e7e475a5e minor fixups for logging 2019-10-18 12:02:36 -04:00
Miroslav Kovar f8a5bc94e5 Pass root logger instead 2019-10-18 12:02:36 -04:00
Miroslav Kovar 6b0b441329 Fix logging bug 2019-10-18 12:02:36 -04:00
Lex Berezhny 1c8aad7a07 fix for encrypted wallet with import read-only channel accounts 2019-10-17 22:28:44 -04:00
Lex Berezhny 7743a6a71d v0.43.2 2019-10-17 13:38:52 -04:00
Lex Berezhny 65a18e78dd publish checks all accounts before forwarding to stream_create/stream_update 2019-10-17 13:38:19 -04:00
Lex Berezhny 12a0cbf4e1 improve preference sync integration test 2019-10-16 17:19:10 -04:00
Lex Berezhny 55c2073813 v0.43.1 2019-10-16 15:50:10 -04:00
Lex Berezhny 39bc8c45c3 test fix 2019-10-16 15:49:10 -04:00
Lex Berezhny 46465f5409 removed WALLET_IS_UNLOCKED condition since it does not take into account multi-wallet 2019-10-16 15:49:10 -04:00
Lex Berezhny 7b5f23e17d incorporate password in the wallet.hash 2019-10-16 15:49:10 -04:00
Lex Berezhny 380efc5ae0 regen docs 2019-10-16 12:06:05 -04:00
Lex Berezhny d1a123997d sync_apply always updates local password if encrypt-on-disk is enabled 2019-10-16 12:06:05 -04:00
Lex Berezhny 722419dffc migrate per-account on disk encryption to encrypt-on-disk preference 2019-10-16 12:06:05 -04:00
Lex Berezhny bcd69f812e sync_apply overwrites local password when encrypt-on-disk is true 2019-10-16 12:06:05 -04:00
Lex Berezhny 1075cd9995 regenerated api.json 2019-10-16 12:06:05 -04:00
Lex Berezhny e5be1ff157 bug fixes and encrypt-on-disk preference 2019-10-16 12:06:05 -04:00
Lex Berezhny 398fe55083 renamed: test_sync.py -> test_wallet_commands.py 2019-10-16 12:06:05 -04:00
shyba b4ee537f7c Merge pull request #2553 from lbryio/choco_37
pin choco py to 37 on win build
2019-10-16 12:11:06 -03:00
Victor Shyba b0ad30a5d8 pin choco py to 37 on win build 2019-10-16 11:48:33 -03:00
Lex Berezhny 26d17e2e69 updated api.json docs 2019-10-15 12:12:35 -04:00
Lex Berezhny 43b619b2c6 release script fix for PRs with null body 2019-10-14 19:09:32 -04:00
Lex Berezhny 0e4426e084 v0.43.0 2019-10-14 13:07:15 -04:00
Jack Robison a27943ea14 update time_to_first_byte.py 2019-10-14 12:59:59 -04:00
Lex Berezhny b8b24da292 encrypt/decrypt/lock/unlock commands return True 2019-10-14 12:57:09 -04:00
Lex Berezhny 45eef03576 fixes 2019-10-14 12:57:09 -04:00
Lex Berezhny 270f77df24 wallet encryption 2019-10-14 12:57:09 -04:00
Lex Berezhny dc580b9eff wip wallet encrypt/decrypt API 2019-10-14 12:57:09 -04:00
Lex Berezhny fed7e94d8d added wallet send command 2019-10-14 10:43:54 -04:00
Lex Berezhny 9bb0ca2f0d async def 2019-10-13 22:32:09 -04:00
Lex Berezhny 3d82becadc utxo release applies to all accounts in wallet 2019-10-13 22:32:09 -04:00
Lex Berezhny e2b92c99ef added wallet balance command 2019-10-13 20:06:11 -04:00
Lex Berezhny fea7275148 mypy fixes 2019-10-12 20:23:29 -04:00
Lex Berezhny 64581fc81b pylint 2019-10-12 20:23:29 -04:00
Lex Berezhny 37ae302fc6 timestamped top-level wallet preferences 2019-10-12 20:23:29 -04:00
Alex Grin c51cc02a87 Merge pull request #2527 from lbryio/faster_trending
only do trending calculations near the tip of the chain
2019-10-11 13:20:35 -04:00
Thomas Zarebczan 6c92fbd8de add oga and ogv mime types
https://gingertech.net/2008/09/05/new-ogg-mime-types-ratified/#targetText=The%20new%20Ogg%20MIME%20types,I%20files%20for%20backwards%20compatibility.
2019-10-11 11:40:36 -04:00
Alex Grintsvayg 065215cc27 only do trending calculations near the tip of the chain 2019-10-11 10:51:50 -04:00
StrikerRUS eb8a88dbd6 refactored integration folder as well 2019-10-11 10:31:54 -04:00
StrikerRUS 62e3258dda revert spaces fixes 2019-10-11 10:31:54 -04:00
StrikerRUS 9959d2c70d use more specific asserts in torba tests 2019-10-11 10:31:54 -04:00
Miroslav Kovar 19e8b6e11f Add test 2019-10-11 10:12:18 -04:00
Miroslav Kovar 63447d1ec3 Make more readable 2019-10-11 10:12:18 -04:00
Miroslav Kovar 75fac93544 Make --verbose behave as per doc 2019-10-11 10:12:18 -04:00
Alex Grin 23f7faa74c Merge pull request #2502 from lbryio/minor_changes
small clarifying changes
2019-10-10 08:59:47 -04:00
Alex Grintsvayg 284fdb2fcb small clarifying changes 2019-10-09 15:42:32 -04:00
Jack Robison 44b27bbfc8 Merge pull request #2487 from lbryio/fix-connection-reset
handle error if range request transport is reset
2019-10-08 16:20:07 -04:00
Jack Robison 35d2d55a27 handle error if range request transport is reset 2019-10-08 14:37:51 -04:00
Jack Robison 292f1953e1 Merge pull request #2521 from lbryio/resolve-ipv4-only
fix resolving fixed peers from an ipv6 connection
2019-10-08 14:37:49 -04:00
Jack Robison 5cd0828dd3 force ipv4 when resolving a url
-log violating ip addresses
2019-10-08 13:31:06 -04:00
Jack Robison 077281a77a v0.42.2 2019-10-07 13:03:51 -04:00
Jack Robison 7055d250d0 Merge pull request #2503 from lbryio/reuse-loggly
Reuse loggly connection
2019-10-07 12:30:04 -04:00
Jack Robison 8c0196301d Merge pull request #2489 from lbryio/dht_object_cache
DHT object cache revamp
2019-10-07 12:27:07 -04:00
Jack Robison f99aacd9dc improve store_to_peer retry, reduce error verbosity 2019-10-07 12:26:49 -04:00
Jack Robison 43b83bec83 remove dead code 2019-10-07 12:26:49 -04:00
Jack Robison fc1b4658fc re-use loggly connection 2019-10-07 12:26:49 -04:00
Victor Shyba 05a28ea346 handle attempts to add peers with no node id and issue a warning 2019-10-07 13:20:38 -03:00
Victor Shyba 624f986978 handle peer_is_good for unknown node id 2019-10-07 13:20:38 -03:00
Victor Shyba 91a9957918 avoid peer mutability
Mutability can be harmful here, specially on fields used for __hash__
It can also make the cache less efficient
2019-10-07 13:20:38 -03:00
Victor Shyba 8cd18e47e7 get_kad... -> make_kad.. 2019-10-07 13:20:38 -03:00
Victor Shyba 517868f73c fix unit tests 2019-10-07 13:20:38 -03:00
Victor Shyba 18d80f4ed5 1 and 65536 arent valid ports 2019-10-07 13:20:38 -03:00
Victor Shyba 2e2c751d75 check for none on tcp port 2019-10-07 13:20:38 -03:00
Victor Shyba 7aa5e2cbe1 use a dataclass experiment 2019-10-07 13:20:38 -03:00
Victor Shyba 7d6db20081 use cache for dht peer objects 2019-10-07 13:20:38 -03:00
Miroslav Kovar 911ec9444b Decrease the status refresh rate 2019-10-07 11:42:40 -04:00
Miroslav Kovar 2883c6bab3 Fix incorrect counting of total bytes sent / received 2019-10-07 11:42:40 -04:00
OrginalS f5fed54195 Punctuation
Added missing punctuation and changed existing to more fitting one.
2019-10-07 10:58:13 -04:00
Rishav Sharma 91b1446375 Grammar correction 2019-10-07 10:50:14 -04:00
Lex Berezhny 56df145052 updating/abanding streams/channels works across all accounts 2019-10-07 09:57:22 -04:00
Lex Berezhny 9e873f77f9 bug fixes related to cross-account / wallet operations 2019-10-07 09:57:22 -04:00
Alex Grintsvayg 27bbd91635 include fn name and args in jsonrpc error message 2019-10-04 11:25:01 -04:00
Alex Grintsvayg 2c73c80c9f fix formatting 2019-10-04 09:52:29 -04:00
Hugo 3a3a9b5f4e Fix spelling 2019-10-03 21:40:06 -04:00
Lex Berezhny 3f89ed4579 v0.42.1 2019-09-30 12:59:45 -04:00
Lex Berezhny 02b01b89f2 fix for wallet_remove command 2019-09-30 12:57:33 -04:00
Victor Shyba 4a6af0b938 retriable resolve 2019-09-30 12:57:18 -04:00
Alex Grintsvayg 9c5b4e3cc9 change trending window to 6 hours 2019-09-30 11:25:52 -04:00
Victor Shyba b3f087dbc7 fix fastest session for when online but unavailable 2019-09-30 10:58:11 -04:00
Jack Robison aa90257ec7 Merge pull request #2481 from lbryio/fix_new_pylint
Fix new pylint
2019-09-25 10:04:01 -04:00
Victor Shyba 348bbc81ca fix for new pylint in lbry/ 2019-09-24 12:53:23 -03:00
Victor Shyba 4231e7f3ff fix for new pylint in torba 2019-09-24 12:23:34 -03:00
Lex Berezhny 4cbc82619d v0.42.0 2019-09-23 13:50:21 -04:00
Lex Berezhny 1b0d802941 is_change bug fix 2019-09-23 13:49:50 -04:00
Lex Berezhny dc72a1e58d fix wallet_create single_key argument 2019-09-23 13:49:50 -04:00
Lex Berezhny 1d5333c737 integration test fix 2019-09-23 13:49:50 -04:00
Lex Berezhny d289041ff3 regnerated docs 2019-09-23 13:49:50 -04:00
Lex Berezhny f287483397 split wallet creation into wallet_add and wallet_create 2019-09-23 13:49:50 -04:00
Lex Berezhny 1ac3b6c173 wallet_list optionally takes wallet_id 2019-09-23 13:49:50 -04:00
Lex Berezhny 1d090765ba pylint dict comp 2019-09-23 13:49:50 -04:00
Lex Berezhny 6c6050528a windows doesnt have json_group_object sqlite function available, switching to group_concat 2019-09-23 13:49:50 -04:00
Lex Berezhny 835d376ca6 wallet_id arg added to more commands 2019-09-23 13:49:50 -04:00
Lex Berezhny 68e9fd8541 account integration test fix 2019-09-23 13:49:50 -04:00
Lex Berezhny e49f8f8c61 regenerated docs 2019-09-23 13:49:50 -04:00
Lex Berezhny b396322f1e removed default_account and default_wallet on Daemon instance 2019-09-23 13:49:50 -04:00
Lex Berezhny ea434e3469 wallet_id related test fixes 2019-09-23 13:49:50 -04:00
Lex Berezhny 84587ac232 wallet commands and wallet_id argument to other commands 2019-09-23 13:49:50 -04:00
Miroslav Kovar 9fbc83fe9d Always handle claim_name as a string 2019-09-20 14:22:01 -04:00
Victor Shyba 965972a719 match regex from app into ours 2019-09-19 17:30:22 -04:00
Miroslav Kovar 34d19c4a72 Remove superfluous test 2019-09-19 17:16:18 -04:00
Miroslav Kovar 9b0910767d Fix formatting, support null argument only 2019-09-19 17:16:18 -04:00
Miroslav Kovar 32cc077e1f enable to set max_key_fee to null via cli 2019-09-19 17:16:18 -04:00
Miroslav Kovar 1e7bec48b6 Add integration test, remove unit test 2019-09-19 15:21:01 -04:00
Miroslav Kovar c035837a14 Fix failing key payment for null max key fee 2019-09-19 15:21:01 -04:00
Miroslav Kovar cb5eb8983b Fix docopt for claim search 2019-09-18 19:30:58 -04:00
Victor Shyba 9c86cb7954 proper dict formatting on tests 2019-09-18 19:28:11 -04:00
Victor Shyba ef94c1eccc use existing donation_address instead 2019-09-18 19:28:11 -04:00
Victor Shyba 64eaa5e3aa add options to set server description, payment address and daily fee 2019-09-18 19:28:11 -04:00
Thomas Zarebczan 826f584aa8 add Flac mime type 2019-09-18 19:20:09 -04:00
Victor Shyba c5530aee5f refactor tests 2019-09-18 19:19:34 -04:00
Victor Shyba 817f01593e call the correct method on is_valid_address 2019-09-18 19:19:34 -04:00
Jack Robison 79727f0e97 Merge pull request #2459 from lbryio/test-sqlite-error-handling
Test sqlite error handling
2019-09-18 10:16:31 -04:00
Jack Robison abac82c1c6 skip test_fetchall_prevents_sqlite_misuse on travis
-print how many attempts the race condition took to trigger
2019-09-17 12:44:43 -04:00
Jack Robison 02c51c6059 typing 2019-09-17 12:09:19 -04:00
Jack Robison 46b45d0387 gotta fetch 'em all 2019-09-17 12:09:19 -04:00
Jack Robison e6c549c457 remove redundant query from sync_missing_blobs 2019-09-17 12:09:19 -04:00
Jack Robison 1aa2983216 fix db errors during shutdown 2019-09-17 12:09:19 -04:00
Jack Robison a095f89c81 remove redundant test 2019-09-17 12:09:19 -04:00
Jack Robison 287902f01b test fetchall prevents SQLITE_MISUSE 2019-09-17 12:09:19 -04:00
Jack Robison 7dc3e4e7a1 test verbosity 2019-09-17 12:09:19 -04:00
Jack Robison 0b31aad901 test binding errors vs sqlite_misuse race errors 2019-09-17 12:09:19 -04:00
Alex Grintsvayg 1791d8e4ee rebased oleg's PR. closes #2446 2019-09-17 11:04:34 -04:00
Lex Berezhny b7c2054a66 mypy fixes 2019-09-17 09:08:29 -04:00
Lex Berezhny 81f16d7061 refactored address_list API to be more detailed 2019-09-17 09:08:29 -04:00
Lex Berezhny 5c151a3a09 v0.41.0 2019-09-16 13:24:11 -04:00
Lex Berezhny 3c46c6cbb9 fix mypy type issue 2019-09-16 13:21:45 -04:00
Lex Berezhny d31f09c174 client wallet db versioning 2019-09-16 13:21:45 -04:00
Lex Berezhny 063eeb34ec fix channel import test to wait for address subscription tasks to finish before looking up channel 2019-09-16 13:21:45 -04:00
Lex Berezhny f06192601c dont fail when claim is not decodable 2019-09-16 13:21:45 -04:00
Lex Berezhny 091221ba2c added wallet/constants.py 2019-09-16 13:21:45 -04:00
Lex Berezhny 7c22111ae0 channel certificate and channel claim can be different accounts, also JSON RPC responses now include a field showing if the channel certificate is available 2019-09-16 13:21:45 -04:00
Alex Grin daac3eed09 Merge pull request #2458 from lbryio/wallet_server_default_coin
default wallet server to LBC coin
2019-09-16 12:34:54 -04:00
Alex Grintsvayg 32ee7feea8 download wallet snapshot on first run 2019-09-16 12:24:44 -04:00
Alex Grintsvayg 2026848a77 default wallet server to LBC coin 2019-09-16 12:01:38 -04:00
Jack Robison a7b205603c Merge pull request #2447 from lbryio/file_encoder
use json_file_encoder for managed streams (file list)
2019-09-13 11:21:36 -04:00
Victor Shyba 291d14f2d4 move as_dict to json_encoder 2019-09-12 14:08:37 -03:00
Victor Shyba 4a68137cfa remove dead code 2019-09-12 14:08:37 -03:00
Jack Robison b2f0374426 Merge pull request #2452 from lbryio/fix-sqlite-misuse
Fix SQLITE_MISUSE
2019-09-12 11:57:23 -04:00
Jack Robison 5fcca6ed0c feedback
-add comment for the fetchall() that if removed will cause SQLITE_MISUSE
2019-09-10 15:03:17 -04:00
Jack Robison a05c44ad1e fix SQLITE_MISUSE
fixes https://github.com/lbryio/lbry-sdk/issues/2164
2019-09-09 22:38:53 -04:00
Jack Robison f9880c8931 test sqlite binding error 2019-09-09 21:46:04 -04:00
Lex Berezhny b876896628 v0.40.1 2019-09-09 13:00:12 -04:00
Victor Shyba 375c2a3363 fix rare edge cases on network 2019-09-09 12:25:16 -04:00
Lex Berezhny 5dbc749000 v0.40.0 2019-09-03 12:36:00 -04:00
Lex Berezhny 36fbd91280 generate docs 2019-09-03 12:00:29 -04:00
Lex Berezhny d2ad3e7ff1 initial kv store 2019-09-03 12:00:29 -04:00
shyba 072f1f112e Merge pull request #2418 from lbryio/no_chdir
sync and connection issues
2019-09-03 11:53:30 -03:00
Victor Shyba 6c79f98d3c connect directly on test server 2019-09-03 11:29:02 -03:00
Victor Shyba 8edebbb684 fix tests for transaction show 2019-09-02 15:43:33 -03:00
Victor Shyba 4839ca631a make restricted default 2019-09-02 15:10:16 -03:00
Victor Shyba 5018fe90de can also be protocolerror 2019-09-01 00:02:07 -03:00
Victor Shyba 379144bcfe fix restriction rules, minor refactor, translate error from transaction not found into returning none 2019-08-31 17:39:03 -03:00
Victor Shyba d2cd0ece5f restore multi server, improve sync concurrency 2019-08-31 06:37:29 -03:00
Victor Shyba 10b7ccaa92 fix unordered history inconsistency 2019-08-31 06:37:29 -03:00
Victor Shyba 925eb618de failing test for unordered mempool 2019-08-31 06:37:29 -03:00
Victor Shyba 96800de052 switch tests and fixes + support for multiple servers 2019-08-31 06:37:29 -03:00
Victor Shyba 9df659b647 no chdir 2019-08-31 06:37:29 -03:00
Lex Berezhny aec0bd5d11 fix intermittent bug where claim metadata was cleared out 2019-08-30 09:44:30 -04:00
Lex Berezhny c1c9c2c578 regenerate api.json 2019-08-28 12:55:19 -04:00
Lex Berezhny c739fa3ca1 fixed typo in daemon API doc 2019-08-26 13:22:42 -04:00
Lex Berezhny 355dfcb6c1 v0.39.3 2019-08-26 13:06:26 -04:00
Oleg Silkin 8f7c291e33 Adds comment_hide functionality 2019-08-26 12:43:07 -04:00
Jack Robison 71fbb3df24 Merge pull request #2414 from lbryio/fix-connection-errors-reloaded
Switch to the fastest wallet server and fix connection errors
2019-08-26 11:13:20 -04:00
Victor Shyba 9cfa9b2c57 use only the main session, dont cancel on connectionerror 2019-08-26 11:17:39 -03:00
Victor Shyba b9b8178e30 use a semaphore to avoid false timeouts on large syncs 2019-08-26 11:17:39 -03:00
Victor Shyba 089cefb77b make it debug for now, there are false positives 2019-08-26 11:17:39 -03:00
Victor Shyba ee9ea1faab get history only from main server 2019-08-26 11:17:39 -03:00
Victor Shyba 22b2cf8b0c warn when out of sync 2019-08-26 11:17:39 -03:00
Victor Shyba 10bd3c9dda fix misc reconnect scenarios 2019-08-26 11:17:39 -03:00
Victor Shyba 6c3147fcf4 simple address subs instead of batch 2019-08-26 11:17:39 -03:00
Jack Robison 5e8d169c50 debug 2019-08-26 11:17:39 -03:00
Jack Robison 8657a35fd7 switch_to_fastest 2019-08-26 11:17:39 -03:00
Jack Robison 97c0b60a22 logging 2019-08-26 11:17:39 -03:00
Lex Berezhny 8a284d6c49 metrics reporting fix for wallet servers 2019-08-26 09:39:42 -04:00
Jack Robison ff4b2f2dec Merge pull request #2411 from lbryio/fix-offline-connection-errors
Fix misc errors when going offline/online
2019-08-22 13:48:02 -04:00
Jack Robison aedce256af fix wallet status silently breaking when offline
-update wallet status to use real latency values and the full list of connected servers
2019-08-22 13:11:17 -04:00
Jack Robison 48b4b97132 fix uncaught exchange rate manager errors when going offline 2019-08-22 13:11:17 -04:00
Jack Robison 6b1f88df66 fix status breaking (and loggly) when the internet connection is lost 2019-08-22 13:11:17 -04:00
shyba d246c46227 Merge pull request #2397 from lbryio/unpack_leftovers
fix unpack error known leftovers
2019-08-22 13:41:25 -03:00
Victor Shyba 7fdd9dc293 fix annoying assertEquals warning 2019-08-21 22:03:24 -03:00
Victor Shyba 202861a900 fix unpack leftovers 2019-08-21 22:03:24 -03:00
Victor Shyba 4c41e7cb29 check headers bounds on access 2019-08-21 22:03:24 -03:00
Jack Robison 92823dcea9 Merge pull request #2407 from lbryio/temporary_speedup_ci
travis: remount /tmp as tmpfs
2019-08-21 20:08:27 -04:00
Victor Shyba 795a5227bc travis: remount /tmp as tmpfs 2019-08-21 20:05:39 -03:00
Alex Grintsvayg bbd4bb99b7 committing my claim search test script 2019-08-21 13:28:30 -04:00
Alex Grin f7443061ef less verbose logging 2019-08-21 09:54:10 -04:00
Jack Robison 416c3c16ab v0.39.2 2019-08-20 20:37:20 -04:00
Jack Robison a11956ece0 Merge pull request #2398 from lbryio/retriable_batcheable_networking
Improve wallet server selection
2019-08-20 20:10:05 -04:00
Jack Robison c4e0c659a7 fix missing perf_counter 2019-08-20 19:22:22 -04:00
Jack Robison f6ba09b171 mypy 2019-08-20 15:41:27 -04:00
Jack Robison b827881024 mypy 2019-08-20 15:35:06 -04:00
Jack Robison 89e395b5f4 pylint 2019-08-20 15:26:00 -04:00
Jack Robison c590954eb9 include connection latency in server selection 2019-08-20 15:11:01 -04:00
Jack Robison a5af3866fd time.time -> time.perf_counter 2019-08-20 14:56:13 -04:00
Jack Robison 1bef56f030 disconnect from duplicate servers (same ip, different dns) 2019-08-20 14:55:24 -04:00
Jack Robison 9045737504 track response time for server.version only 2019-08-20 14:54:55 -04:00
Jack Robison 0e48772c84 track connection latency 2019-08-20 14:45:53 -04:00
Jack Robison 72690b5cff raise asyncio.TimeoutError 2019-08-20 14:19:50 -04:00
Jack Robison 07bae26fd3 close connection and reraise on CancelledError 2019-08-20 14:16:56 -04:00
Victor Shyba 7b0e4617d3 fix db test stability 2019-08-20 04:11:04 -03:00
Victor Shyba af797296ed rolling averages on response time 2019-08-19 16:50:24 -03:00
Victor Shyba c826c7da0d take load into consideration when picking fastest 2019-08-19 14:02:30 -03:00
Victor Shyba 20af1396b9 refactor stop 2019-08-19 14:02:30 -03:00
Victor Shyba f567aca532 retry and batch requests, fix some loose ends 2019-08-19 14:02:30 -03:00
Lex Berezhny c60b658443 v0.39.1 2019-08-19 13:02:16 -04:00
Jack Robison fde8c34088 Merge pull request #2401 from lbryio/check-blob-peers-pingable
UDP ping blob peers to prioritize those who can be connected to
2019-08-19 13:01:40 -04:00
Jack Robison deb6a07160 fix test 2019-08-19 12:51:17 -04:00
Jack Robison 0a66e1b24c logging 2019-08-19 12:29:13 -04:00
Jack Robison 819a551b77 Merge pull request #2402 from lbryio/aioupnp-0.0.14
bump `aioupnp` requirement to 0.0.14, fixes upnp errors blocking startup
2019-08-19 12:08:22 -04:00
Jack Robison 5119b71035 prevent attempting to download from oneself 2019-08-19 12:08:17 -04:00
Jack Robison 33945e14d3 use _value_producer in peer_list to match the downloader behavior 2019-08-19 12:08:17 -04:00
Jack Robison 1d9705fb17 reuse blob protocol object instead of transport 2019-08-19 12:08:17 -04:00
Jack Robison 1949937b5e bump aioupnp requirement to 0.0.14 2019-08-19 10:51:20 -04:00
Lex Berezhny 956e21269e monitor wallet server block transitions and report if new block is excessively delayed 2019-08-19 09:52:37 -04:00
Alex Grin ff73418fc1 Merge pull request #2371 from lbryio/basenetwork_refactor
refactor basenetwork so each session takes care of itself
2019-08-16 10:19:53 -04:00
shyba 76d40f44d8 Merge pull request #2395 from lbryio/lbrycrd17
add support for lbrycrd 17
2019-08-14 11:41:13 -03:00
Victor Shyba ab6944b7b0 wallet server: dont fail to start on segwit txs 2019-08-14 10:26:02 -03:00
Victor Shyba 4d741bd200 update lbrycrd to 17, add option to disable segwit for now 2019-08-13 18:07:15 -03:00
Victor Shyba f3a163b382 fix names and types 2019-08-12 18:16:58 -03:00
Victor Shyba 011b7f090f add urgent reconnect when api is called to bypass retry delay 2019-08-12 18:16:58 -03:00
Victor Shyba 4ead92cfbe fixes from review 2019-08-12 18:16:58 -03:00
Victor Shyba 9ee2f30df4 timeout instead of cancel + minor fixes 2019-08-12 18:16:58 -03:00
Victor Shyba 4a749f6c38 on timeout, restore latency to max 2019-08-12 18:16:58 -03:00
Victor Shyba cf924373f6 make stream test an asyncio test 2019-08-12 18:16:58 -03:00
Victor Shyba d1f5b25418 merge repeated subscription events 2019-08-12 18:16:57 -03:00
Victor Shyba 822f53c888 remove unused things 2019-08-12 18:16:57 -03:00
Victor Shyba f04c5ee902 fix test reliability 2019-08-12 18:16:57 -03:00
Victor Shyba 022d60ec2c both exceptions can be raised, depending on when the call happens 2019-08-12 18:16:57 -03:00
Victor Shyba 8fce374cae race condition where the request is sent and connection lost right after 2019-08-12 18:16:57 -03:00
Victor Shyba e78e2738ef basenet refactor 2019-08-12 18:16:57 -03:00
Victor Shyba 5728493abb refactor basenetwork so each session takes care of itself 2019-08-12 18:16:57 -03:00
Lex Berezhny af1c8ec35c adding missing comma in wallet_server_monitor 2019-08-12 14:16:47 -04:00
Lex Berezhny cb84abfb88 v0.39.0 2019-08-12 13:01:16 -04:00
Jack Robison df1e349470 Merge pull request #2385 from lbryio/blob-exchange-timeouts
Blob exchange server timeouts
2019-08-12 12:29:40 -04:00
Jack Robison 357ce881a6 more blob exchange tests 2019-08-12 10:47:28 -04:00
Jack Robison aa417f5410 test max request size 2019-08-12 10:47:28 -04:00
Jack Robison 0ca29b9518 test idle timeout 2019-08-12 10:47:28 -04:00
Jack Robison 22db04068e add server side timeouts for idle blob exchange connections and slow transfers 2019-08-12 10:47:28 -04:00
Jack Robison 65e74281e0 max request size 2019-08-12 10:47:28 -04:00
Lex Berezhny 46d89929ba fixup balance tests and other stuff 2019-08-12 10:22:14 -04:00
Lex Berezhny 1c941a355b regenerate api docs 2019-08-12 10:22:14 -04:00
Lex Berezhny 0d6248fbeb fix torba tests 2019-08-12 10:22:14 -04:00
Lex Berezhny 98d4d00f96 most commands now work across all accounts 2019-08-12 10:22:14 -04:00
Jack Robison e2d618f472 formatting in time_to_first_byte.py
-pending claim_type being usable in claim_search, don't try downloading channel claims in time_to_first_byte.py
2019-08-12 00:49:33 -04:00
Jack Robison 79d643db6c Merge pull request #2384 from lbryio/faster_file_list
Faster file list
2019-08-11 22:41:15 -04:00
Victor Shyba 13891330d9 check blob hashes from completed_blob_hashes 2019-08-11 19:11:13 -03:00
Jack Robison 6fd0c2bf77 Merge pull request #2379 from lbryio/add-wallet-server-to-ttfb-analytics
Add `connected_servers` field to wallet status and improve analytics for wallet related download errors
2019-08-09 14:18:03 -04:00
Jack Robison b8b42a1fd2 add error_message to ttfb analytics, update tests 2019-08-09 12:55:01 -04:00
Jack Robison fba1b28615 add connected_servers field to wallet status
-add `wallet_server` to time to first bytes analytics event
2019-08-09 12:54:38 -04:00
Lex Berezhny 98828387e3 slow queries saved to metabase 2019-08-06 20:41:14 -04:00
Lex Berezhny 1c12247b71 updated wallet_server_monitor to update slack 2019-08-06 17:44:31 -04:00
Lex Berezhny ee10ebd5d7 renamed lbry/scripts/wallet_server_metabase_analytics.py -> lbry/scripts/wallet_server_monitor.py 2019-08-06 16:07:45 -04:00
Lex Berezhny 9937ad6e85 v0.38.7 2019-08-05 13:40:49 -04:00
Jack Robison 60ce393531 remove comment 2019-08-05 13:39:38 -04:00
Jack Robison bc33cad999 add test for connection failure analytics 2019-08-05 13:39:38 -04:00
Jack Robison 4261b50d9d include connection failure count in ttfb analytics 2019-08-05 13:39:38 -04:00
Jack Robison e2b3141536 typing 2019-08-05 13:39:38 -04:00
Thomas Zarebczan 266aa1b15d update delete>abandon 2019-08-05 12:42:25 -04:00
Oleg Silkin b57a7475c5 Fixes comment docs & adds wallet component requirement on comment funcs 2019-08-05 12:42:25 -04:00
Victor Shyba 8cf48eb5f7 move granular balances logic to account method 2019-08-05 11:08:33 -04:00
Victor Shyba 35541bbbb9 new api format, hide subtotals by default 2019-08-05 11:08:33 -04:00
Victor Shyba 59424a3e73 add query to get tips summary for granular balances 2019-08-05 11:08:33 -04:00
Victor Shyba 2056888667 implement tips_received tips_sent, wip, in python for now 2019-08-05 11:08:33 -04:00
Victor Shyba 1e54785742 add granular balances 2019-08-05 11:08:33 -04:00
Lex Berezhny a58ed6f331 wallet server metabase analytics script 2019-08-05 09:23:47 -04:00
Lex Berezhny 9a79b71d9f removed dart 2019-08-04 20:57:05 -04:00
Lex Berezhny e9098887fb script to check channel signature 2019-08-01 17:27:54 -04:00
Jack Robison 9e8a3ef7fc Merge pull request #2359 from lbryio/fix-error-on-null-fee
Fix dust transaction error on zero amount content fee
2019-07-30 11:53:04 -04:00
Jack Robison 221585908a re-raise unexpected resolve error during download as a ResolveError 2019-07-29 17:56:30 -04:00
Jack Robison f4c111f04c handle null content fee 2019-07-29 17:56:24 -04:00
Alex Grin de12e25450 v0.38.6 2019-07-29 15:11:18 -04:00
Oleg Silkin 2545ebbece Adds unittests & fixes comment_client functions 2019-07-29 14:55:42 -04:00
Oleg Silkin 14ec6ce43b Adds unit-testing for delete comment signatures 2019-07-29 14:55:42 -04:00
Oleg Silkin 0cac1e65aa Imports comment_client and uses dot notation for rest of modules 2019-07-29 14:55:42 -04:00
Oleg Silkin 1e4d57ab83 Simplifies signing function into one to make API symmetric 2019-07-29 14:55:42 -04:00
Oleg Silkin 49b42b7e03 Simplifies 2019-07-29 14:55:42 -04:00
Oleg Silkin 89c0b5dbdf adds comment abandon function 2019-07-29 14:55:42 -04:00
Lex Berezhny 8138675808 added index for claim_type searches 2019-07-29 14:55:07 -04:00
Jack Robison 726129836a Merge pull request #2323 from lbryio/refactor_ttfb
refactor and fix ttfb for new api
2019-07-28 22:28:02 -04:00
Jack Robison 356b3c85e5 use app default order_by 2019-07-28 21:42:04 -04:00
Jack Robison 1a0680ead9 update claim_search_performance.py, add new test cases shown by time_to_first_byte.py 2019-07-28 21:22:07 -04:00
Jack Robison c876d891fa use perf_counter 2019-07-28 21:18:29 -04:00
Jack Robison a08232f164 fix claim search kwargs in time to first byte script 2019-07-28 21:18:13 -04:00
Victor Shyba 0f8cf46626 refactor and fix ttfb for new api 2019-07-28 17:33:37 -04:00
Alex Grin 95017b5fc8 v0.38.5 2019-07-25 13:22:54 -04:00
Jack Robison 1374d337d8 Merge pull request #2348 from lbryio/probemon
Probe if peers can be connected to during blob downloads to more quickly skip bad peers
2019-07-25 13:21:42 -04:00
Jack Robison 35e7cc685f fix logging error 2019-07-24 22:12:51 -04:00
Victor Shyba 2ac4883bda cleanup and fix prober 2019-07-24 22:32:20 -03:00
Victor Shyba 5a7212aebb gotta probe 'em all 2019-07-24 20:26:38 -03:00
Lex Berezhny 3f39cc2955 v0.38.4 2019-07-24 18:02:03 -04:00
Lex Berezhny 8a5bfe884a save channel keys into wallet before broadcasting 2019-07-24 18:00:06 -04:00
Lex Berezhny 046dd53913 add index to filter by fee_amount and order by release_time 2019-07-24 14:27:59 -04:00
Alex Grintsvayg d2f174635c add script to monitor wallet servers for slow queries 2019-07-24 13:36:22 -04:00
Lex Berezhny 01a760d8f0 v0.38.3 2019-07-23 17:44:38 -04:00
Victor Shyba 9215d2c9f9 log that checkpoint succeeded 2019-07-23 17:44:13 -04:00
Victor Shyba 0bb09ae91e use checkpoint to speed up 2019-07-23 17:44:13 -04:00
Victor Shyba afeff61268 fix repair and tests 2019-07-23 17:44:13 -04:00
Victor Shyba 6f5a595975 check from final size instead of partials 2019-07-23 17:44:13 -04:00
Victor Shyba 8a69ec062e download headers to disk and repair instead of connecting each 2019-07-23 17:44:13 -04:00
Victor Shyba 45fb673e49 dont fail when s3 is out of reach 2019-07-23 17:44:13 -04:00
Lex Berezhny 93b5f1809e revert to v0.38.2 so we can release v0.38.3 again 2019-07-23 17:43:04 -04:00
Alex Grin ff05eaae48 v0.38.3 2019-07-23 11:14:20 -04:00
Alex Grintsvayg b48b67ed10 fix aws key 2019-07-23 11:14:00 -04:00
Alex Grintsvayg 3ead6f63b6 add lbry-ci bot auth token 2019-07-23 11:04:10 -04:00
Alex Grin 77235f0ad8 v0.38.2 2019-07-22 17:45:43 -04:00
Alex Grintsvayg f6a80ba929 Revert "v0.38.2"
This reverts commit 295ceb90d5.
2019-07-22 17:45:12 -04:00
Alex Grin 295ceb90d5 v0.38.2 2019-07-22 13:31:20 -04:00
Victor Shyba df41a82b23 fix integration test 2019-07-22 11:31:45 -04:00
Victor Shyba 34500e07c6 change picking logic to probe the session before considering it a good session 2019-07-22 11:31:45 -04:00
Alex Grintsvayg 1e6097e3b2 more spv dns entries 2019-07-22 11:24:12 -04:00
Lex Berezhny 521facedc4 script to find max server load 2019-07-22 00:15:05 -04:00
Lex Berezhny de80a2ba97 fixed 0 results when only filtering by lots of common tags 2019-07-21 19:44:10 -04:00
Lex Berezhny 3afd615447 dont run common tag optimization strategy if there are 5 or more common tags provided 2019-07-20 15:45:15 -04:00
Lex Berezhny 87e52bec0a successfuly run indexes after full wallet server sync 2019-07-20 15:45:15 -04:00
Lex Berezhny b89f04b1f1 fix claim integration tests 2019-07-20 15:45:15 -04:00
Lex Berezhny e4f802e5f4 when resolving inside channel break a tie of claims with same effective_amount by using height 2019-07-20 15:45:15 -04:00
Lex Berezhny 16fc2c456f moved executing create index queries outside of a tx 2019-07-20 15:45:15 -04:00
Lex Berezhny ef7656ae98 removed some dead code 2019-07-20 15:45:15 -04:00
Lex Berezhny ad2df4871e popular tags are indexed and use different search strategy than less popular tags 2019-07-20 15:45:15 -04:00
Jack Robison 8d5f50b695 improve claim search test script to show sql for nearly interrupted queries 2019-07-19 14:17:21 -04:00
Lex Berezhny bbc056eef0 added better stats to sqlite_perf_test.py 2019-07-19 11:26:50 -04:00
Jack Robison 82fd3c632e another claim search performance test case 2019-07-19 10:50:37 -04:00
Jack Robison dbae939808 claim search performance test 2019-07-19 10:42:38 -04:00
Jack Robison c2dcacb0de more test cases 2019-07-19 10:37:15 -04:00
Lex Berezhny 8f45618cd0 added improved effective_amount index covering release_time 2019-07-18 23:24:00 -04:00
Lex Berezhny 29a87c850c fix sqlite_perf_test 2019-07-18 19:54:38 -04:00
Lex Berezhny 395518ff76 ver updates to new metrics 2019-07-18 19:54:38 -04:00
Lex Berezhny 08da763327 more metrics cleanup 2019-07-18 19:54:38 -04:00
Lex Berezhny d7ec3c182d tuples 2019-07-18 19:54:38 -04:00
Lex Berezhny ffe6418392 fix metrics json response test 2019-07-18 19:54:38 -04:00
Lex Berezhny 88adf3ee27 oops 2019-07-18 19:54:38 -04:00
Lex Berezhny f406db85dc pylint again 2019-07-18 19:54:38 -04:00
Lex Berezhny d5f8af471a pylint 2019-07-18 19:54:38 -04:00
Lex Berezhny 046459a72d refactored websockets and metrics 2019-07-18 19:54:38 -04:00
Lex Berezhny 49b0f59790 added metrics module 2019-07-18 19:54:38 -04:00
Lex Berezhny 71551058ce query interpolation 2019-07-18 19:54:38 -04:00
Jack Robison 2a9e6473eb Merge pull request #2310 from lbryio/2293
set file status to completed only when all bytes were written
2019-07-18 13:21:41 -04:00
Victor Shyba 576ef9c4ed remove unused variable 2019-07-17 18:54:21 -03:00
Victor Shyba 4d64fca9de fix memory blob showing as completed/verified 2019-07-17 18:54:21 -03:00
Victor Shyba 8b1307e1ca file status completed set only if all bytes were written 2019-07-17 18:54:21 -03:00
Jack Robison eae4ed7b27 Merge pull request #2317 from lbryio/new_2282
torba client networking: improve network timeout handling and fix spurious logs on shutdown
2019-07-17 17:49:59 -04:00
Victor Shyba 6997ea608d fix reconnect and timeouts 2019-07-16 23:59:28 -03:00
Alex Grintsvayg d97d738880 remove RCs from release tool 2019-07-16 21:35:21 -04:00
Lex Berezhny f72b07d0f4 fixup 2019-07-16 21:28:36 -04:00
Lex Berezhny b89916a3d7 guarded dict access 2019-07-16 21:15:39 -04:00
Lex Berezhny e4aeb66257 claim_list doesnt blow up 2019-07-16 21:08:51 -04:00
Jack Robison 578f8f4062 Merge pull request #2320 from lbryio/interrupt-slow-queries
Interrupt and log slow wallet server queries
2019-07-16 20:50:37 -04:00
Jack Robison 8fbf82f4ea filter mutually exclusive tags 2019-07-16 20:05:49 -04:00
Jack Robison 478ae7e657 add claim search performance script 2019-07-16 20:05:49 -04:00
Jack Robison d319761483 interrupt slow queries and log them for debugging
- add QUERY_TIMEOUT_MS env variable

- return sqlite timeout errors, jsonrpc error code -32000
2019-07-16 20:05:45 -04:00
Lex Berezhny 16fb716872 thicker chart line 2019-07-16 17:10:02 -04:00
Lex Berezhny b8d9d097d8 graph taller 2019-07-16 15:23:45 -04:00
Lex Berezhny ec79a1d0e8 improved graph ui 2019-07-16 15:21:41 -04:00
Lex Berezhny 31284e0c5e dont run empty queries 2019-07-16 13:57:22 -04:00
Lex Berezhny 8e64b1840d added errors key to metrics tracking 2019-07-16 13:39:17 -04:00
Lex Berezhny dd9bf04e35 default to HOST for WEBSOCKET_HOST if not set 2019-07-16 12:05:43 -04:00
Thomas Zarebczan 76a27fdef1 str > list for a few fields 2019-07-16 00:59:06 -04:00
Victor Shyba f96443515a optimize header check 2019-07-16 00:58:36 -04:00
Lex Berezhny 0f1bde2274 pylint 2019-07-16 00:57:53 -04:00
Lex Berezhny 6fb94e445f ver gets cache hit info 2019-07-16 00:57:53 -04:00
Lex Berezhny e02a4e44c0 working resolve/search caching 2019-07-16 00:57:53 -04:00
Lex Berezhny 71dc882d5d indexes update 2019-07-16 00:57:53 -04:00
Lex Berezhny ae4a29d41c prevent duplicate tags 2019-07-16 00:57:53 -04:00
Lex Berezhny d27a3b6421 support for UN M.49 region codes in language 2019-07-16 00:57:53 -04:00
Lex Berezhny 03a58cf50a added lrucache to search/resolve 2019-07-16 00:57:53 -04:00
Lex Berezhny f43e370e89 refactor 2019-07-16 00:57:53 -04:00
Lex Berezhny 45bdb9023a ui to add servers 2019-07-16 00:57:53 -04:00
Lex Berezhny 6d7aa93157 sqlite_perf_test uses lbry.wallet.server.db.reader 2019-07-16 00:57:53 -04:00
Lex Berezhny d083910255 refactored metrics gathering for wallet server 2019-07-16 00:57:53 -04:00
Lex Berezhny db3f9af174 refactored metrics gathering for wallet server 2019-07-16 00:57:53 -04:00
Lex Berezhny 7d2ad5e4ed added TRACK_METRICS env variable to torba wallet server 2019-07-16 00:57:53 -04:00
Lex Berezhny 84d3771984 more detailed stress testing metrics and graphs 2019-07-16 00:57:53 -04:00
Lex Berezhny 1eebe006ae pylint 2019-07-16 00:57:53 -04:00
Lex Berezhny 0327ce53a6 uncomment imports 2019-07-16 00:57:53 -04:00
Lex Berezhny 85442c1d07 websocket admin connection to get query metric data 2019-07-16 00:57:53 -04:00
Lex Berezhny a6ed51b7a7 websocket admin connection to get query metric data 2019-07-16 00:57:53 -04:00
Lex Berezhny dc8b4b5252 instrument wallet.server.db.reader with timer metrics 2019-07-16 00:57:53 -04:00
Lex Berezhny 8ca74d3de2 env variables to set websocket host and port 2019-07-16 00:57:53 -04:00
LBRY bot 1923e6f152 v0.38.1 2019-07-15 12:02:18 -04:00
Lex Berezhny 0d621e085b v0.38.1rc1 2019-07-15 11:58:13 -04:00
Victor Shyba 8749c78622 close daemon connector on server close to avoid errors on tests 2019-07-14 00:33:57 -04:00
Victor Shyba 510d88fff1 remove unused checksum, add typing 2019-07-13 01:21:26 -04:00
Victor Shyba d9460dcd9e refactor and clean up 2019-07-13 01:21:26 -04:00
Victor Shyba d0607b6fec make header component use torba headers 2019-07-13 01:21:26 -04:00
Victor Shyba 2481305a1a make integrity check faster 2019-07-13 01:21:26 -04:00
Victor Shyba 9990889cce add repair ability to baseheaders 2019-07-13 01:21:26 -04:00
Victor Shyba 14f743c493 show status where expected 2019-07-13 01:21:26 -04:00
Victor Shyba df8fe2e3ef bump header checksum 2019-07-13 01:21:26 -04:00
Victor Shyba 906fea972e remove lost warning message 2019-07-13 01:21:26 -04:00
Victor Shyba 04a82d5c23 add make test 2019-07-13 01:21:26 -04:00
Victor Shyba f2944fd33b if no headers, best block hash is genesis 2019-07-13 01:21:26 -04:00
Victor Shyba dcb1e53aba remove unused network command 2019-07-13 01:21:26 -04:00
Victor Shyba a4d8508b40 update remote height on demand 2019-07-13 01:21:26 -04:00
Victor Shyba f109aa1f73 make header access synchronous for now 2019-07-13 01:21:26 -04:00
Victor Shyba f09c18539b add test for concurrent header access 2019-07-13 01:21:26 -04:00
Lex Berezhny c5aeac6898 sqlite performance script measures sql time inside each process individually 2019-07-11 20:10:28 -04:00
Lex Berezhny b81c801f2d sqlite perf test script 2019-07-11 19:20:58 -04:00
Lex Berezhny f3cedf2f44 added windows build for ver 2019-07-11 17:20:50 -04:00
Jack Robison 910999e7e4 fix dart bug
fixes `The argument type 'Utf8Decoder' can't be assigned to the parameter type 'StreamTransformer<Uint8List, dynamic>'.` on `flutter run`
2019-07-11 17:15:51 -04:00
Lex Berezhny 5bceb06fc5 load tester supports changing host, port and payload 2019-07-11 16:13:16 -04:00
Lex Berezhny cc098f3090 refactored wallet server db reader to support thread pool and process pool 2019-07-11 14:04:49 -04:00
Lex Berezhny a73c5aec27 pylint 2019-07-11 14:04:49 -04:00
Lex Berezhny 68b29d9529 multi-process query executor 2019-07-11 14:04:49 -04:00
Lex Berezhny 57f069d750 added ability to customize SessionManager to torba 2019-07-11 14:04:49 -04:00
Lex Berezhny 67e12bc590 ver app updates 2019-07-11 14:04:49 -04:00
Lex Berezhny ff2a08747b added lbry dart 2019-07-11 14:04:49 -04:00
Lex Berezhny 3791d22713 added ver 2019-07-11 14:04:49 -04:00
Lex Berezhny 638ecafb69 no longer using virtualenv for travis windows build 2019-07-09 18:42:34 -04:00
Alex Grintsvayg 5176feed81 let daemon be shut down before it has fully started. fixes #2231 2019-07-09 16:55:48 -04:00
Lex Berezhny 8c63033461 tx parsing speedup and faster wallet sync 2019-07-09 02:20:56 -04:00
Lex Berezhny 8c878f8e25 claim search uses different tag filtering approaches depeding on query type 2019-07-08 16:33:24 -04:00
Victor Shyba c1c7e30ec9 test boundaries, make limit a constant 2019-07-08 12:03:24 -04:00
Victor Shyba 28ae7f81e9 closes 2281 - fix too many sql variables + tests 2019-07-08 12:03:24 -04:00
hackrush 71de4d769d Merge pull request #2299 from lbryio/gitignore-update-mac
added .DS_Store to .gitignore
2019-07-08 20:50:04 +05:30
hackrush f502944c5b added .DS_Store to .gitignore 2019-07-08 19:40:39 +05:30
Lex Berezhny 99d21e93f3 added --no_totals argument to claim_search 2019-07-07 23:51:32 -04:00
Lex Berezhny 350e610f15 handle invalid fee in old JSON claims 2019-07-06 20:53:03 -04:00
Thomas Zarebczan 4f208b6b31 Merge pull request #2286 from osilkin98/2279-fix
Fixes corner case on comment validation for pending channel
2019-07-03 14:14:47 -04:00
Thomas Zarebczan dcb781a57e Merge pull request #2287 from lbryio/aiohttp
use utils.aiohttp for comments
2019-07-03 14:12:22 -04:00
Thomas Zarebczan 09db71ae7f Update comment_client.py 2019-07-02 23:39:27 -04:00
Thomas Zarebczan 60dc43340d use util.aiohttp
Fix certificate errors
2019-07-02 18:38:43 -04:00
Oleg Silkin 256b10bd7d Merge branch 'master' into 2279-fix 2019-07-02 12:01:00 -04:00
Oleg Silkin 77adcaeb50 Fixes corner case where channel is pending confirmation 2019-07-02 11:58:24 -04:00
Jack Robison abf2ca40a2 Merge pull request #2273 from lbryio/fix-session-bloat
Fix wallet server session bloat from unhandled socket errors
2019-07-01 12:18:57 -04:00
Jack Robison d5183123fa fix uvloop being ignored 2019-07-01 11:42:16 -04:00
Jack Robison 8b0d96acba reuse lbrycrd connector 2019-07-01 11:40:13 -04:00
Jack Robison 66122a45ed remove zombie sessions on uncaught socket errors 2019-07-01 11:36:26 -04:00
Jack Robison a2953ac9ac test session bloat from timed out sockets 2019-07-01 11:36:26 -04:00
Jack Robison c51db5294f typing 2019-07-01 11:36:26 -04:00
Jack Robison 8cba43bfed add method_counts field to getinfo 2019-07-01 11:36:23 -04:00
Jack Robison 4b1c7aa6c5 fix getinfo command 2019-07-01 10:41:36 -04:00
Jack Robison 14c1e95841 Merge pull request #2275 from lbryio/disable-bandwidth-tracking
add `track_bandwidth` config setting to disable bandwidth usage in `status`, add `total_sent` and `total_received` fields to connection status
2019-07-01 10:39:50 -04:00
Jack Robison 4f5a4d9ce0 add track_bandwidth config setting to disable bandwidth usage in status 2019-07-01 09:54:06 -04:00
Lex Berezhny 82f2ad21e1 added IGNORE to insert into trend query 2019-06-28 16:02:54 -04:00
Jack Robison 8a5b803788 add first 32260 blocks of bitcoin headers for torba unit tests 2019-06-28 15:09:49 -04:00
Lex Berezhny 86b46ebf19 fix for has_channel_signature in claim_search 2019-06-26 19:19:10 -04:00
Lex Berezhny ca999053ae update lbrycrd to v0.12.4.1 2019-06-26 12:40:26 -04:00
Lex Berezhny 8b152f3c96 check for valid fee.currency and fee.amount before processing 2019-06-26 10:44:33 -04:00
Lex Berezhny c68bfbdf2e v0.38.0 2019-06-25 16:27:01 -04:00
Jack Robison 7e8180fefc fix attribute error if availability_response field is missing 2019-06-25 16:22:02 -04:00
Jack Robison 305226a3bd fix database migration bugs 2019-06-25 16:09:24 -04:00
Jack Robison c8752b03c1 travis checksums 2019-06-25 16:09:24 -04:00
Lex Berezhny 8195deebe3 updated generated docs 2019-06-25 15:34:32 -04:00
Lex Berezhny 88f2156aeb update install to new testing tools and directory structure 2019-06-25 13:49:54 -04:00
Lex Berezhny b1982a81dc updated release script to add --start-tag option 2019-06-25 12:50:13 -04:00
Lex Berezhny 18e19213f4 fix fee_amount default on wallet server 2019-06-25 11:04:32 -04:00
Lex Berezhny 6fb67b86f5 added spv4-spv6 2019-06-25 10:42:59 -04:00
Lex Berezhny a4c15769d0 v0.38.0rc10 2019-06-25 02:01:42 -04:00
Lex Berezhny 894e86d3ee fix intermittent test_queries failure in torba due to non-unique TX generated 2019-06-25 01:33:04 -04:00
Lex Berezhny 9712ef8abf cache Homebrew 2019-06-25 01:33:04 -04:00
Lex Berezhny 908107d24f cache virtualenv directory on windows 2019-06-25 01:33:04 -04:00
Lex Berezhny d667efa9de remove docker wine based windows build process scripts 2019-06-25 01:33:04 -04:00
Lex Berezhny e649d7219b stage: build 2019-06-25 01:33:04 -04:00
Lex Berezhny 1f5698ec86 build using travis windows container 2019-06-25 01:33:04 -04:00
Jack Robison 15341aa80e fix setting the external ip from upnp 2019-06-24 15:39:31 -04:00
Lex Berezhny 3f4895bd40 v0.38.0rc9 2019-06-23 23:42:29 -04:00
Jack Robison 103c93a06c update aioupnp to 0.0.13 2019-06-23 23:39:45 -04:00
Lex Berezhny 21f02d3e6d fix torba test_sync test reliability 2019-06-23 22:40:41 -04:00
Lex Berezhny 9ce93a1da6 added --not_channel_ids and --has_channel_signature 2019-06-23 20:24:35 -04:00
Lex Berezhny d0e1468acd channel migration better handles missing or invalid certificates 2019-06-23 13:46:02 -04:00
Lex Berezhny 8a5bb239c5 fix working directory in windows build 2019-06-23 11:31:46 -04:00
Lex Berezhny 816cde32a9 fix support reorg error 2019-06-23 10:56:59 -04:00
Lex Berezhny 2b514a9df7 fix github release upload 2019-06-23 00:38:02 -04:00
Lex Berezhny ef39ba26a0 v0.38.0rc8 2019-06-23 00:08:41 -04:00
Lex Berezhny 59ca3d3d16 take_over_height fix 2019-06-22 23:24:56 -04:00
Lex Berezhny 80ff60f0aa added take_over_height to claim search / resolve output 2019-06-22 23:24:56 -04:00
Lex Berezhny 488c2e2f17 searching by fee_amount and fee_currency 2019-06-22 23:24:56 -04:00
Lex Berezhny 9e11fc1e99 dont error when claim search gets invalid channel 2019-06-22 23:24:56 -04:00
Lex Berezhny a8b590ecbb add tag normalization on storage and on search 2019-06-22 23:24:56 -04:00
Lex Berezhny 8ca8591ed6 wrong test method reverted 2019-06-22 23:24:56 -04:00
Lex Berezhny e393314423 fix for claims_in_channel not updating on abandoned claim 2019-06-22 23:24:56 -04:00
Lex Berezhny c54373bd9c moved enum into script to avoid weird import conflict with Python built-in enum module 2019-06-22 23:24:56 -04:00
Alex Grin f459b6833a Update INSTALL.md 2019-06-21 14:13:14 -04:00
Lex Berezhny 0a6eb76542 cleaned up make idea 2019-06-21 11:39:35 -04:00
Lex Berezhny e6d42b5319 .idea directory now requires running make idea first 2019-06-21 11:26:58 -04:00
Lex Berezhny 0d2804ce55 travis windows build path fixes 2019-06-21 03:27:22 -04:00
Lex Berezhny 8efdd9d29a popd directory before windows build 2019-06-21 03:27:22 -04:00
Lex Berezhny 27c36a4dce fix get_spendable_utxos 2019-06-21 03:27:22 -04:00
Lex Berezhny 2f74611844 pylint 2019-06-21 03:27:22 -04:00
Lex Berezhny d8265add2d added coin selection strategy for both prefer_confirmed and only_confirmed 2019-06-21 03:27:22 -04:00
Lex Berezhny 01cd02e4c5 import fix 2019-06-21 03:27:22 -04:00
Lex Berezhny 2b98f85961 fix module paths in .idea/lbry-sdk.iml 2019-06-21 03:27:22 -04:00
Alex Grintsvayg 338d1bd8e7 add coin selection strategy config 2019-06-21 03:27:22 -04:00
Lex Berezhny 7dedaeeef7 remaining lbry -> lbry-sdk changes 2019-06-20 23:13:57 -04:00
Lex Berezhny 54830b8968 renamed repo to lbry-sdk 2019-06-20 23:03:48 -04:00
Lex Berezhny 41b866c1ee fix linux/windows build 2019-06-20 22:50:39 -04:00
Lex Berezhny fe7224e537 residual fixes after lbrynet -> lbry move 2019-06-20 22:50:39 -04:00
Lex Berezhny f5e38f6e23 updated imports after moving lbrynet -> lbry 2019-06-20 22:50:39 -04:00
Lex Berezhny 063335377e updated lbry.iml 2019-06-20 22:50:39 -04:00
Lex Berezhny ee6f511126 moved lbrynet -> lbry 2019-06-20 22:50:39 -04:00
Lex Berezhny 75574486d3 simplify lbry.iml 2019-06-20 22:50:39 -04:00
Lex Berezhny 03a09f0f42 ignore changes to files in .idea 2019-06-20 22:50:39 -04:00
Lex Berezhny 075b3ec3d2 added default .idea files 2019-06-20 22:50:39 -04:00
Lex Berezhny 3d5aa6a44b travis fixes 2019-06-20 22:50:39 -04:00
Lex Berezhny 8350b42c1b require python 3.7 2019-06-20 22:50:39 -04:00
Lex Berezhny 7061a62e88 set correct directory in Docker build 2019-06-20 22:50:39 -04:00
Lex Berezhny 133fceeee5 fixing docker build 2019-06-20 22:50:39 -04:00
Lex Berezhny 38331b8918 travis fixups 2019-06-20 22:50:39 -04:00
Lex Berezhny 5bbba0b65a updated windows build 2019-06-20 22:50:39 -04:00
Lex Berezhny 07a63b2a0a moved lbry.png into lbry 2019-06-20 22:50:39 -04:00
Lex Berezhny 9ddc70fc81 deleted .github templates 2019-06-20 22:50:39 -04:00
Lex Berezhny 2b6f5fde13 moved more lbry files into lbry dir 2019-06-20 22:50:39 -04:00
Lex Berezhny d4b006656d torba integration test 2019-06-20 22:50:39 -04:00
Lex Berezhny b1e70723c7 travis coverage combine fix 2019-06-20 22:50:39 -04:00
Lex Berezhny 1ec1ad19f8 pin to mypy==0.701 2019-06-20 22:50:39 -04:00
Lex Berezhny b96b27ed91 travis ci install mypy 2019-06-20 22:50:39 -04:00
Lex Berezhny 8b145cb1d8 combined .travis.yml 2019-06-20 22:50:39 -04:00
Lex Berezhny 85d7adc0b5 moved lbrynet into lbry directory 2019-06-20 22:50:39 -04:00
Lex Berezhny d530599b5d Merge remote-tracking branch 'torba/move-to-lbry' 2019-06-20 14:04:30 -04:00
Lex Berezhny 16215d5b96 preparing for moving torba into lbry sdk repo 2019-06-20 13:45:46 -04:00
Jack Robison db685e5807 bump torba requirement 2019-06-19 14:23:38 -04:00
Jack Robison 190304344c v0.5.7 2019-06-19 10:55:57 -04:00
Alex Grintsvayg 24e8a3c62e more accurate coin selection strategy name 2019-06-19 10:39:11 -04:00
Lex Berezhny 571b02f381 v0.38.0rc7 2019-06-19 06:44:26 -04:00
Lex Berezhny 36e3f1b3ec integration test fix 2019-06-19 06:33:12 -04:00
Lex Berezhny 564d6bdc5d more refactoring 2019-06-19 06:33:12 -04:00
Victor Shyba 1e2b040f75 simplify validation, move to fee.update 2019-06-19 06:33:12 -04:00
Victor Shyba 5876d5c295 cover more edge cases, add tests 2019-06-19 06:33:12 -04:00
Victor Shyba 54e53932bd fix bug: fee address defaults respect replace 2019-06-19 06:33:12 -04:00
Victor Shyba c9b3c2e6aa change test to use preview=True on broadcast 2019-06-19 06:33:12 -04:00
Victor Shyba 912c102d9a fix updating fee address to claim address, fix testing for malformed claims 2019-06-19 06:33:12 -04:00
Victor Shyba 2b7cbf9759 get without fee address defaults pays to claim address 2019-06-19 06:33:12 -04:00
Oleg Silkin 1cc6b2e31e More integration unittests and mock server fixes 2019-06-19 06:25:26 -04:00
Oleg Silkin 7fa797712a Adds comment signature unit testing 2019-06-19 06:25:26 -04:00
Oleg Silkin 636f36ebad Moves get_channel,get_stream out of Class into their own functions 2019-06-19 06:25:26 -04:00
Oleg Silkin 8e0888b846 Fixes comment functions 2019-06-19 06:25:26 -04:00
Oleg Silkin ce5dd5022b Adds Comment Signing and Validation 2019-06-19 06:25:26 -04:00
Lex Berezhny 4a24cc0095 pylint & mypint fixes 2019-06-19 06:12:57 -04:00
Alex Grintsvayg f2bd0edc51 add option to only use confirmed utxos 2019-06-19 06:12:57 -04:00
Thomas Zarebczan 756d17bada update wallet servers
Not sure where spv3 will be yet. We can probably repurpose lbryumx2/3/4 once we go live, so we'll make sure to add those before release.
2019-06-19 02:53:41 -04:00
Lex Berezhny 81e061af30 added missing raise SystemExit() 2019-06-19 02:53:22 -04:00
Alex Grintsvayg a60cd1d475 separate try/except for media server 2019-06-19 02:53:22 -04:00
Lex Berezhny f78323e0fb update torba version 2019-06-19 02:17:08 -04:00
Lex Berezhny fd6cbe0f6c reorgs do not hang the wallet server 2019-06-19 02:17:08 -04:00
Lex Berezhny c30e905e19 undo backup_txs method signature change 2019-06-19 01:49:47 -04:00
Lex Berezhny f5650c1b13 minor refactor of reorg integration test 2019-06-19 01:49:47 -04:00
Lex Berezhny ecebbc86c6 fixed reorg issue with headers 2019-06-19 01:49:47 -04:00
Jack Robison bfeed40bfe Merge pull request #2247 from lbryio/fix-unable-to-serialize-response
paginated deterministically shuffled find_value
2019-06-18 23:25:12 -04:00
Jack Robison 2ad22d7d19 paginated deterministically shuffled find_value
- fixes https://github.com/lbryio/lbry/issues/2244
- reduce the max DHT datagram size to 1400 bytes
- truncate `contacts` field of find_value response datagrams to the k closest (8)
- truncate peers in find_node response datagrams to the 2k closest (16)
- remove `contacts` field from find_value responses beyond `page` 0 (the first/default)
- deterministically shuffle the peers for a blob in a find_value response
- add optional `page` argument to `find_value` and `p` field to find_value responses containing the number of pages of k peers for the blob
- test one blob being announced by 150 different peers to one peer
- speed up pylint and remove some disabled checks
2019-06-18 22:52:39 -04:00
Lex Berezhny 262735c219 --blocking 2019-06-18 19:32:49 -04:00
Alex Grin 3abc98acc8 Update PULL_REQUEST_TEMPLATE.md 2019-06-18 09:17:54 -04:00
Alex Grin eaa8ba814e Update ISSUE_TEMPLATE.md 2019-06-17 14:06:01 -04:00
Jack Robison 588568752b Merge pull request #2217 from lbryio/chunked-reflector-response
fix handling chunked reflector server responses
2019-06-11 14:05:06 -04:00
Lex Berezhny 794eac0519 0.5.6 2019-06-10 23:49:04 -04:00
Victor Shyba b75139662b refactor pick_fastest_session 2019-06-10 23:25:38 -04:00
Victor Shyba 9e24f4ca54 fix mypy, add types 2019-06-10 23:25:38 -04:00
Victor Shyba 1b5281c23e connect to multiple servers 2019-06-10 23:25:38 -04:00
Lex Berezhny be15601a67 pylint 2019-06-10 01:44:07 -04:00
Lex Berezhny 35cd42bfea ledger.wait() accept timeout argument 2019-06-10 01:44:07 -04:00
Jack Robison a30e2daf05 handle chunked responses from reflector server 2019-06-07 17:26:31 -04:00
Jack Robison adb7edd3b2 test chunked reflector server response 2019-06-07 17:26:31 -04:00
Jack Robison 06a9770504 Merge pull request #2216 from lbryio/fix-connection-status-startup-error
fix connections status raising an error during startup
2019-06-07 13:55:54 -04:00
Jack Robison 0da43d3e05 fix connections status raising an error during startup 2019-06-07 12:14:05 -04:00
Lex Berezhny 077dbb3138 better handle claims which fail to decode 2019-06-07 08:42:17 -04:00
Jack Robison 3b8fd65fa9 Merge pull request #2212 from lbryio/add-torba-version-to-version-rpc
update `version` docstring, add `torba_version`
2019-06-05 20:59:58 -04:00
Jack Robison d0ddaa5518 update version docstring, add torba_version 2019-06-05 20:33:21 -04:00
Jack Robison a60ab97552 Merge pull request #2211 from lbryio/pin-aioupnp
pin aioupnp requirement to 0.0.12
2019-06-05 19:41:45 -04:00
Jack Robison 078d09976c pin aioupnp requirement 2019-06-05 18:50:36 -04:00
Jack Robison 3770558694 Merge pull request #2210 from lbryio/fix-wine-build
fix torba version on travis
2019-06-05 18:46:24 -04:00
Jack Robison 4409b8c4f8 fix torba version on travis 2019-06-05 18:19:56 -04:00
Lex Berezhny 03e48c0968 0.5.5 2019-06-05 17:07:36 -04:00
Lex Berezhny f5b7aaac18 v0.38.0rc6 2019-06-04 22:32:33 -04:00
Lex Berezhny 6dbb6b9079 fix claim expiration calculation 2019-06-04 22:28:35 -04:00
Lex Berezhny 16b774a69e add normalized to claims_in_channel index to make update claim faster 2019-06-04 22:28:35 -04:00
Lex Berezhny 72ea19532e added index specifically for claim_in_channel computation 2019-06-04 22:28:35 -04:00
Lex Berezhny ad24946af3 add improved profiling to validate_channel_signatures 2019-06-04 22:28:35 -04:00
Lex Berezhny b676907fd7 added index for claim expiration_height 2019-06-04 22:28:35 -04:00
Lex Berezhny 6965971613 delete claims and re-calculate winner when claim is expired 2019-06-04 22:28:35 -04:00
Lex Berezhny 6651111522 pylint 2019-06-04 18:44:21 -04:00
Lex Berezhny d8630a02d9 Strings setting works like Server from CLI 2019-06-04 18:44:21 -04:00
Lex Berezhny 69345b4b0e integrationt tests fixed 2019-06-04 16:04:46 -04:00
Lex Berezhny f23aea9951 better handling for claims in abandoned channels 2019-06-04 16:04:46 -04:00
Lex Berezhny f96c46e84a claim search fixes for filtering with valid_channel_signatures 2019-06-04 16:04:46 -04:00
Victor Shyba 3ea92c0d6e pylint 2019-06-04 11:04:47 -04:00
Victor Shyba 8e43e40810 fix reconnect and improve test 2019-06-04 11:04:47 -04:00
Alex Grintsvayg bae2939943 change a few instances of lbry.io to lbry.com. fixes #2135 2019-06-04 10:23:04 -04:00
Alex Grintsvayg 7063138e3c small improvements from our cli investigation 2019-06-04 09:39:33 -04:00
Lex Berezhny 798ba9acf8 v0.38.0rc5 2019-06-04 01:20:32 -04:00
Jack Robison c8dd1987e6 add connection status 2019-06-04 01:20:05 -04:00
Lex Berezhny 8d2c9e5785 claim_type/stream_type/media_type claim search integration test 2019-06-04 00:51:25 -04:00
Lex Berezhny 9dbf47916b add public_key_id to channel integration tests 2019-06-03 21:56:17 -04:00
Lex Berezhny 565ef5e454 docopt doc string fix 2019-06-03 21:56:17 -04:00
Lex Berezhny 9f1f2b1781 claim search with --public_key_id 2019-06-03 21:56:17 -04:00
Lex Berezhny 76df239d1a start command accepts server lists consistent with other commands 2019-06-03 21:22:24 -04:00
Thomas Zarebczan 86ee6a78f5 add MKV to mime types 2019-06-03 19:51:48 -04:00
Victor Shyba a14b91cdda refactor test and make it faster 2019-06-03 16:04:15 -04:00
Victor Shyba 08609e9760 remove unnecessary call to ledger stop on client tests 2019-06-03 16:04:15 -04:00
Victor Shyba ca0a3dbf9e minor cleanups on test spurious exceptions 2019-06-03 16:04:15 -04:00
Victor Shyba 9d0b9f0ded improve server picking tests 2019-06-03 16:04:15 -04:00
Victor Shyba 8911af58cf pick fastest server on client start 2019-06-03 16:04:15 -04:00
Lex Berezhny 4fb6f2bb27 review fixes 2019-06-03 06:51:09 -04:00
Oleg Silkin ec039825e8 Fixes non-subscribtable object error 2019-06-03 06:51:09 -04:00
Oleg Silkin a10f09d169 Updates signature return field 2019-06-03 06:51:09 -04:00
Oleg Silkin 76409600e1 Fixes stuff up 2019-06-03 06:51:09 -04:00
Oleg Silkin d06a0f876a Patches comment server interface in the SDK 2019-06-03 06:51:09 -04:00
Oleg Silkin 8e857de62a Fixes non-subscribtable object error 2019-06-03 06:51:09 -04:00
Oleg Silkin 3d37f7c735 Updates signature return field 2019-06-03 06:51:09 -04:00
Oleg Silkin 552b94bc65 Fixes stuff up 2019-06-03 06:51:09 -04:00
Oleg Silkin 17164601a1 Patches comment server interface in the SDK 2019-06-03 06:51:09 -04:00
Lex Berezhny 26dd6d2ac7 claim search test fix 2019-05-30 21:57:49 -04:00
Lex Berezhny 5d5e36a2c2 short_url and canonical_url returned from wallet server use original claim name instead of normalized 2019-05-30 21:32:35 -04:00
Lex Berezhny 77afbcded6 wallet server, skip table join for count(*) query 2019-05-30 21:07:07 -04:00
Lex Berezhny ae11d72f66 added claim.effective_amount index on wallet server 2019-05-30 16:13:46 -04:00
Lex Berezhny d2773f9713 v0.38.0rc4 2019-05-29 22:48:28 -04:00
Lex Berezhny a7fba569cd update tests to use valid_channel_signatures argument 2019-05-29 22:47:33 -04:00
Lex Berezhny 0559c64038 fixed claim search bug related to signed claims 2019-05-29 22:47:33 -04:00
Lex Berezhny 5ab2754bc6 shutdown daemon during tests teardown 2019-05-29 19:27:05 -04:00
Lex Berezhny 6bfb8384e7 channel import/export working tests 2019-05-29 19:27:05 -04:00
Lex Berezhny 0156aa8b00 channel import/export wip 2019-05-29 19:27:05 -04:00
Lex Berezhny ad411cc157 trying to fix rebase 2019-05-29 19:27:05 -04:00
hackrush 6d462ad6d8 channel export import 2019-05-29 19:27:05 -04:00
Lex Berezhny d23920cc61 added get_public_key methods 2019-05-29 01:38:53 -04:00
Lex Berezhny 1b35cef77a fix wallet.server.test_sqldb test 2019-05-28 20:51:22 -04:00
Lex Berezhny a357208a77 cleaned up with passing tests 2019-05-28 20:51:22 -04:00
hackrush 7e1f8a56d5 Changed txo.private_key to be an object 2019-05-28 20:51:22 -04:00
hackrush 37665f2df7 Review Fixes 2019-05-28 20:51:22 -04:00
hackrush 8f3b596385 Add consolidated field to migration result 2019-05-28 20:51:22 -04:00
hackrush e45c6fe42c Only store pubkey-private key pairs 2019-05-28 20:51:22 -04:00
hackrush 56edf62cb9 Temporary fix for github.com/PyCQA/pylint/issues/2820 2019-05-28 20:51:22 -04:00
hackrush d367ff6ac1 Update tests and code 2019-05-28 20:51:22 -04:00
hackrush 7a78d49e13 Store public-private mappings in wallet 2019-05-28 20:51:22 -04:00
Lex Berezhny 4dae1ae05f regenerate docs 2019-05-27 23:18:26 -04:00
Lex Berezhny 9004f240e6 bug fixes 2019-05-27 23:18:26 -04:00
Lex Berezhny 1848462ad9 v0.38.0rc3 2019-05-26 14:04:45 -04:00
Lex Berezhny 786efa06dd fixed resolve to work with partial claim_ids 2019-05-26 14:01:06 -04:00
Lex Berezhny b6811d55cf no longer log skipped claim insertions on wallet server, happens too often 2019-05-26 14:01:06 -04:00
Lex Berezhny 42dc0eccd6 wallet server channel key change error 2019-05-26 14:01:06 -04:00
Lex Berezhny 29bd936181 wallet server tracks claim signature 2019-05-26 14:01:06 -04:00
Lex Berezhny 9d97b90ff4 pylint 2019-05-26 14:01:06 -04:00
Lex Berezhny 370b34f860 canonical_url takes into account whether channel signature is valid 2019-05-26 14:01:06 -04:00
Lex Berezhny affa46e0f6 initial import of canonical url via the new sql wallet server 2019-05-26 14:01:06 -04:00
Jack Robison 6b93705599 Merge pull request #2181 from lbryio/improve-set-build
improve set_build.py, pin torba version
2019-05-24 19:00:38 -04:00
Jack Robison 925e39c5a8 improve set_build.py, pin torba version
- adds BUILD_COMMIT to lbrynet.build_type
2019-05-24 18:02:55 -04:00
Jack Robison 1efa2c9406 Merge pull request #2180 from lbryio/fix-completed-false-negative
fix false negative with file `completed` field
2019-05-24 17:18:04 -04:00
Jack Robison 4f8bcfcd18 fix false negative with file completed field 2019-05-24 16:47:25 -04:00
Jack Robison 3ed72a215e Merge pull request #2179 from lbryio/fix-range-request-start-offset
start returning bytes for a range request at the requested starting position
2019-05-24 16:46:21 -04:00
Jack Robison 85d94d4ca3 start returning range request bytes at the requested offset 2019-05-24 15:32:40 -04:00
Jack Robison b922ea8e7e Merge pull request #2178 from lbryio/lru-cached-read-blob
add a lru cache for decrypted blobs to minimize redownloading them, add `blob_lru_cache_size` to the config to set the cache size
2019-05-24 13:56:59 -04:00
Jack Robison cb6ddaf276 add blob_lru_cache_size config setting, defaults to 32 2019-05-24 12:02:58 -04:00
Lex Berezhny 1c2029cddf fix startup error when single key account present in wallet 2019-05-22 18:31:49 -04:00
Alex Grintsvayg d8d136f1d8 uniform spacing on docstring 2019-05-22 16:51:14 -04:00
Victor Shyba fedc0e5bc3 add checktrie.py script 2019-05-22 12:20:33 -04:00
Lex Berezhny ee353ca776 0.5.4 2019-05-22 09:35:23 -04:00
Alex Grintsvayg 4fbccb9336 uniform spacing on docstring 2019-05-22 09:23:31 -04:00
Lex Berezhny ef5dd5593f v0.38.0rc2 2019-05-20 21:39:16 -04:00
Lex Berezhny baed974be0 several minor wallet server fixes 2019-05-20 21:35:20 -04:00
Lex Berezhny b2b3509b61 regenerate protobufs 2019-05-20 15:19:14 -04:00
Lex Berezhny 51720d9b73 add MAX() to claimtrie GROUP BY algo to make results consistent across SQLite versions 2019-05-20 15:19:14 -04:00
Lex Berezhny 6abb569ec8 v0.38.0rc1 2019-05-19 23:03:25 -04:00
Lex Berezhny c2fbbadc69 start trending calculation as wallet server sync nears best height 2019-05-19 21:16:19 -04:00
Lex Berezhny b8897223ec added zscore trending in wallet server 2019-05-19 21:16:19 -04:00
hackrush c2c184b4ef Updated to include the environment variables needed in mac 2019-05-18 21:44:49 -04:00
Lex Berezhny 07dbdef3fe dont run test_resolve_lru_cache_doesnt_persist_errors 2019-05-18 21:43:46 -04:00
Lex Berezhny b841fb2f0c fix docopt parser issue 2019-05-18 21:43:46 -04:00
Lex Berezhny 2b75982446 disable trending until its fixed 2019-05-18 21:43:46 -04:00
Lex Berezhny 021ca3efd2 updated doc strings and fixed wallet server claimtrie implementation 2019-05-18 21:43:46 -04:00
Lex Berezhny e56b72cf04 pylint and use torba master 2019-05-18 21:43:46 -04:00
Lex Berezhny 342f485ddf good stuff 2019-05-18 21:43:46 -04:00
Lex Berezhny 8fe16fbfc9 performance 2019-05-18 21:43:46 -04:00
Lex Berezhny 1e3035029e timer fix 2019-05-18 21:43:46 -04:00
Lex Berezhny 5f4c02f836 wallet server sync performance improvements 2019-05-18 21:43:46 -04:00
Lex Berezhny 03a1185e0f pylint 2019-05-18 21:43:46 -04:00
Lex Berezhny 0e1705d19e instrumented most wallet server claim related functions with timer 2019-05-18 21:43:46 -04:00
Lex Berezhny 82c739daa6 abstract _apply_constraints_for_array_attributes for wallet server 2019-05-18 21:43:46 -04:00
Lex Berezhny 4702bea9f0 claim_search --not_tags feature + integration tests 2019-05-18 21:43:46 -04:00
Lex Berezhny 75211b7e84 updated claim_search doc string 2019-05-18 21:43:46 -04:00
Lex Berezhny 9f9273bd82 working claim_search with tags 2019-05-18 21:43:46 -04:00
Lex Berezhny d2608bdf43 fixed wallet/server/test_sqldb.py tests 2019-05-18 21:43:46 -04:00
Lex Berezhny 091bef1ead sequence/amount_order resolve 2019-05-18 21:43:46 -04:00
Lex Berezhny 998ee9feba resolve integration tests fixed 2019-05-18 21:43:46 -04:00
Lex Berezhny 587f0b2ea8 integration tests for jsonrpc_transaction_show fixed 2019-05-18 21:43:46 -04:00
Lex Berezhny 048aa07e80 test_file_commands integration tests fixed 2019-05-18 21:43:46 -04:00
Lex Berezhny 7ca01511f4 fixing integration tests 2019-05-18 21:43:46 -04:00
Lex Berezhny 8dee83aac0 disable cache for integration tests while using custom torba branch 2019-05-18 21:43:46 -04:00
Lex Berezhny 879ae5cc72 .travis.yml 2019-05-18 21:43:46 -04:00
Lex Berezhny 74a9e17178 .travis.yml 2019-05-18 21:43:46 -04:00
Lex Berezhny 93d2173fad chris45 integration test passing 2019-05-18 21:43:46 -04:00
Lex Berezhny a459ee34be test_url unit test fix 2019-05-18 21:43:46 -04:00
Lex Berezhny 2e6297f834 pylint 2019-05-18 21:43:46 -04:00
Lex Berezhny 9accbfcf8b move exception handling into is_signed_by 2019-05-18 21:43:46 -04:00
Lex Berezhny 6946860521 added result schema types 2019-05-18 21:43:46 -04:00
Lex Berezhny 9b8be8682c massive progress 2019-05-18 21:43:46 -04:00
Lex Berezhny 03455310ae rewrote URL parser 2019-05-18 21:43:46 -04:00
Lex Berezhny e5edaed677 removed test code from wallet server daemon.py 2019-05-18 21:43:46 -04:00
Lex Berezhny 5e97215eaa integration tests should clone sqltrie torba branch 2019-05-18 21:43:46 -04:00
Lex Berezhny 266f94bc95 pylint 2019-05-18 21:43:46 -04:00
Lex Berezhny d26caded53 use sqltrie branch of torba 2019-05-18 21:43:46 -04:00
Lex Berezhny 2260608bb6 working sql claimtrie 2019-05-18 21:43:46 -04:00
Lex Berezhny bba4c6e4f4 added __gte sql criteria and pass header to wallet server advance_txs 2019-05-18 00:03:23 -04:00
Lex Berezhny 1d126913e9 added __or and __and to sql query generator 2019-05-18 00:03:23 -04:00
Lex Berezhny b3afee2f86 ignore too-many-statements pylint 2019-05-18 00:03:23 -04:00
Lex Berezhny 31983fbb8e support in db for constraints that reference the same column by appending a # followed by some unique string 2019-05-18 00:03:23 -04:00
Lex Berezhny ba8870823c added network.get_transaction_height 2019-05-18 00:03:23 -04:00
Lex Berezhny 5f1bc01636 swap advance_txs arguments 2019-05-18 00:03:23 -04:00
Lex Berezhny ffa9e508e9 block_processor passes height on to advance_txs 2019-05-18 00:03:23 -04:00
Lex Berezhny cfb051396a sql __in uses proper escaping instead of putting values directly in SQL 2019-05-18 00:03:23 -04:00
Lex Berezhny bb1f42057e v0.37.1 2019-05-17 23:01:47 -04:00
Lex Berezhny 695f6658a0 v0.37.1rc2 2019-05-17 21:43:37 -04:00
Jack Robison eca677a720 Merge pull request #2148 from lbryio/lru-cache-concurrent
Fix resolve internals caching and persisting errors
2019-05-17 19:26:22 -04:00
Jack Robison 51f301785f fix test, add lru_cache_concurrent decorator 2019-05-17 18:57:07 -04:00
Jack Robison 41a6e288aa add test_resolve_lru_cache_doesnt_persist_errors 2019-05-17 18:56:28 -04:00
Jack Robison 97d8d0ef4c Merge pull request #2133 from lbryio/more-wallets
Add missing wallet server
2019-05-17 15:50:10 -04:00
Niko Storni 5baeb5f538 fix examples 2019-05-17 15:19:11 -04:00
Niko Storni cc49c8407c add missing wallet server
switch to .com domain
2019-05-17 15:19:11 -04:00
Jack Robison 3f37963f7f Merge pull request #2139 from lbryio/android-platform-name
fix system name on android
2019-05-17 14:39:20 -04:00
Jack Robison 12eb83f773 fix system name on android 2019-05-17 14:00:25 -04:00
Jack Robison ca1d135ba4 Merge pull request #2138 from lbryio/fix-file-save-doc
fix save_file doc
2019-05-17 13:44:22 -04:00
Jack Robison 107e57e3ff fix docs 2019-05-17 13:11:04 -04:00
Jack Robison 928689a10d Merge pull request #2130 from lbryio/fix-datastore
Fix updating refreshed announcements in the datastore
2019-05-17 12:58:11 -04:00
Jack Robison 6ecc22f2c7 fix refreshing peers in the datastore 2019-05-17 12:22:47 -04:00
Lex Berezhny 079c81f298 v0.37.1rc1 2019-05-17 08:59:10 -04:00
Victor Shyba 8afe178d62 remove supports from resolve 2019-05-17 08:55:39 -04:00
Jack Robison 8777dcee7c v0.37.0 2019-05-13 23:39:22 -04:00
Jack Robison 1cca4a91dd Merge pull request #2128 from lbryio/add-streaming-server-settings
add `streaming_server` and `streaming_get` settings to configure the interface/port of the media server and if it supports the `/get` endpoint
2019-05-13 22:51:53 -04:00
Jack Robison 41abfbdd9b add streaming_server and streaming_get settings
-split rpc runner from range request runner
2019-05-13 22:27:48 -04:00
Jack Robison 46ee65c8f2 Merge pull request #2127 from lbryio/add-streaming-url
add `streaming_url` field to `File` api response type
2019-05-13 21:05:52 -04:00
Jack Robison fd19daad9e add streaming_url to file api response type 2019-05-13 19:05:26 -04:00
Jack Robison 26d183cbab Merge pull request #2125 from lbryio/dht_wip
Remove search junctions, make search retry, some refactoring
2019-05-13 18:02:56 -04:00
Victor Shyba 71a4be02dd use loop to create task 2019-05-13 15:57:03 -03:00
Victor Shyba 124fdb7d31 fixes from review 2019-05-13 14:57:58 -03:00
Victor Shyba 2439743804 probe peer instead of scheduling ping for removal 2019-05-13 14:51:03 -03:00
Victor Shyba 6f79875135 fix test 2019-05-13 14:27:49 -03:00
Victor Shyba b91d2190f4 disable infinite peer search, cleanup logging, tune scores to slow connections 2019-05-13 14:24:13 -03:00
Victor Shyba d024433d1b simplify accumulate task 2019-05-13 13:34:49 -03:00
Victor Shyba 7c4eb416d6 dont clearbanned if there are working peers, dont remove node immediately during search 2019-05-13 13:14:08 -03:00
hackrush 397ebe8428 Added DS_Store to .gitignore 2019-05-13 09:54:19 -04:00
Victor Shyba fb457c820a reorganizing iterative find code 2019-05-13 04:34:39 -03:00
Victor Shyba 18af2dcd4e remove peer immediately instead of probing when its bad during search 2019-05-13 02:52:24 -03:00
Victor Shyba 0075dcc2c0 iterative find probes bad peers + test case for edge scenario Tom found 2019-05-13 02:40:04 -03:00
Victor Shyba 6e327d729b fix test 2019-05-12 04:28:58 -03:00
Victor Shyba 20022d49c1 less verbose dht tests 2019-05-12 04:13:58 -03:00
Victor Shyba 32d1563885 fix handling of invalid token 2019-05-12 04:12:12 -03:00
Victor Shyba f5cef14d8d test get token on announce 2019-05-12 03:55:07 -03:00
Victor Shyba bac7d99b8a add ability to re-join network on disconnect + tests 2019-05-12 03:39:11 -03:00
Victor Shyba 6f06026511 announcer goes idle if no peers in routing table 2019-05-12 01:10:05 -03:00
Victor Shyba f02df86709 fix announce loop when there are no peers to announce 2019-05-12 00:42:19 -03:00
Victor Shyba b7d76fd09f add integration tests, fix bug that made refresh not exclude bad peers 2019-05-11 23:30:05 -03:00
Victor Shyba e675f1387c remove peer junction and some refactoring 2019-05-11 04:58:56 -03:00
Victor Shyba 0da2827c78 fix dht deadlock 2019-05-10 19:54:59 -03:00
Jack Robison 309cfc840c v0.37.0rc7 2019-05-10 15:34:38 -04:00
Jack Robison 68ee31b62a Merge pull request #2123 from lbryio/stop-timed-out-downloads
Stop timed out downloads
2019-05-10 15:29:33 -04:00
Jack Robison ad2da24d64 improve test 2019-05-10 14:57:04 -04:00
Jack Robison 648dbcc36b add timeout to blob downloads after the head blob, stop timed out streams 2019-05-10 14:50:01 -04:00
Victor Shyba e59e96190e fix peer search using inexistent variable, fix test to cover it 2019-05-10 13:49:35 -04:00
Jack Robison 7d2f592bb5 Merge pull request #2107 from lbryio/fix-update-file-we-dont-have
fix stream_update failing on metadata only updates for files we don't have
2019-05-10 12:28:53 -04:00
Jack Robison 1a182bc9bd bytes/str 2019-05-10 11:50:20 -04:00
Jack Robison 324cbdcdb2 only save the content claim if stream_hash isn't None
-stream_hash being none means we don't have the file in the database
2019-05-10 11:50:20 -04:00
Lex Berezhny 6c0c0ee322 added logging to publish command 2019-05-10 11:43:34 -04:00
Jack Robison 1e54271d8a Merge pull request #2105 from lbryio/fix-store-content-fee
fix store content fee
2019-05-09 18:47:30 -04:00
Jack Robison 54b9ed6cb2 fix store content fee 2019-05-09 18:14:54 -04:00
Lex Berezhny 2626a28e6d 0.5.3 2019-05-09 18:10:27 -04:00
Lex Berezhny fb2d7cbe28 pylint 2019-05-09 18:05:17 -04:00
Lex Berezhny 874b3faf8c use a cache when loading TXs in db.get_txos() 2019-05-09 18:05:17 -04:00
Lex Berezhny 558a02d187 correctly format lat/long and fee amounts in JSON RPC response 2019-05-09 16:19:49 -04:00
Jack Robison 3567b331c0 v0.37.0rc6 2019-05-09 11:52:34 -04:00
Jack Robison 9e4cf3c995 Merge pull request #2048 from lbryio/timeout_settings
Adjust default max_connections_per_download and peer_connect_timeout settings
2019-05-09 11:14:59 -04:00
Thomas Zarebczan db1d547b19 adjust the default max_connections_per_download and peer_connect_timeout 2019-05-09 10:38:32 -04:00
Jack Robison c31efd780b docs 2019-05-09 10:32:07 -04:00
Jack Robison 7cdb7987e3 Merge pull request #2102 from lbryio/shutdown_fast
Set a shutdown timeout for TCPSite to avoid hanging on exit
2019-05-09 10:24:13 -04:00
Victor Shyba ec07037e88 set shutdown_timeout on TCPSite 2019-05-09 01:01:07 -03:00
Lex Berezhny 9bb7b0571e fix integrat tests 2019-05-09 00:00:39 -04:00
Lex Berezhny d98162a118 stream_update --replace also clears the channel 2019-05-09 00:00:39 -04:00
Jack Robison c5ae59bff7 Merge pull request #2100 from lbryio/junction_fixes
DHT junction fixes for cases that can lock up
2019-05-08 23:52:54 -04:00
Victor Shyba efbf2f49a9 fix generator junction 2019-05-08 23:02:38 -03:00
Victor Shyba 77c69f661d add locking cases scenarios 2019-05-08 23:02:38 -03:00
Victor Shyba 2e09d2adfa close search on transport close 2019-05-08 23:02:38 -03:00
Jack Robison 8620a0e424 Merge pull request #2095 from lbryio/add-content-fee-and-saved-file-to-db
add saved_file and content_fee columns to file table
2019-05-08 20:37:56 -04:00
Jack Robison e2c123456b write decrypted blob data from run_in_executor 2019-05-08 16:07:32 -04:00
Jack Robison c9be79f062 prevent re-migrating 10 to 11 2019-05-08 15:13:07 -04:00
Jack Robison a2a1ef34ab fix tests 2019-05-08 14:55:09 -04:00
Jack Robison 761a2b235c set written_bytes, download_path, file_name, and download_directory to None if the file doesn't exist 2019-05-08 14:11:44 -04:00
Jack Robison 3ca2fee592 use stream size from claim if available 2019-05-08 14:11:39 -04:00
Jack Robison a480b2d25f fix race when closing blob client protocol after getting a response 2019-05-08 12:21:11 -04:00
Jack Robison d7032b12d7 add saved_file and content_fee columns to file table
-drop not null constraints for file_name and download_directory

-add migrator
2019-05-08 10:26:06 -04:00
Jack Robison 84381ff76c Merge pull request #2093 from lbryio/fix_announce_on_storage
Refactor announcer + fixups for last db migrator code
2019-05-07 10:46:12 -04:00
Victor Shyba 007dd43861 re-raise cancelled errors 2019-05-07 11:10:49 -03:00
Victor Shyba 1bc4e4b702 typo on logging, announcements logging is info, not warning 2019-05-07 11:10:49 -03:00
Victor Shyba b91f27219d use fetchall as there is an update while iterating 2019-05-07 11:10:49 -03:00
Victor Shyba dc4560cc9a refactor announcer 2019-05-07 11:10:49 -03:00
Victor Shyba 84b471d486 limit batch size on announcer test so it covers batching logic 2019-05-07 11:10:49 -03:00
Victor Shyba c6a3c05a0c add missing migrator line 2019-05-07 11:10:49 -03:00
Jack Robison dd665a758d Merge pull request #2065 from lbryio/streaming-bug-fixes
Streaming bug fixes
2019-05-07 10:10:39 -04:00
Jack Robison 32dc1297ec fix infinite loop 2019-05-06 18:46:26 -04:00
Jack Robison bef244ba77 dont make duplicate files 2019-05-06 18:46:26 -04:00
Jack Robison 3c231d6215 handle get with file_name/download_directory/save_file arg for a stream that is already running 2019-05-06 18:46:26 -04:00
Jack Robison a212cf6ba4 set content_fee attribute 2019-05-06 18:46:26 -04:00
Jack Robison bc060f1378 pay fee after downloading the sd blob/adding to the db 2019-05-06 18:46:26 -04:00
Jack Robison 22c701fd50 fix completed field for an item in file list 2019-05-06 18:46:26 -04:00
Jack Robison 4f0e8fce6e add optional download_directory argument to get 2019-05-06 18:46:26 -04:00
Jack Robison f642cfe9dd fix resuming save tasks on startup 2019-05-06 18:46:26 -04:00
Jack Robison 24e073680b add connection id workaround 2019-05-06 18:46:26 -04:00
Jack Robison b2f63a1545 fix tests 2019-05-06 18:46:26 -04:00
Jack Robison f506b3e6d4 tests 2019-05-06 18:46:26 -04:00
Jack Robison f7412514b3 fix partial content 0 padding 2019-05-06 18:46:26 -04:00
Jack Robison bd6a609b30 fix logging 2019-05-06 18:46:26 -04:00
Jack Robison 4e32b69d1d don't set running streams as stopped on startup 2019-05-06 18:46:26 -04:00
Jack Robison 1116c7f29e logging 2019-05-06 18:46:26 -04:00
Jack Robison d80c671cf2 look victor 2019-05-06 18:46:26 -04:00
Jack Robison 14b12cbea2 block /stream and /get requests on the stream manager starting 2019-05-06 18:46:26 -04:00
Jack Robison 1f7feafb67 force close open streaming requests and api calls on shutdown 2019-05-06 18:46:26 -04:00
Jack Robison cbe689ea7f logging 2019-05-06 18:46:26 -04:00
Jack Robison 9099ee2e8e fix/refactor starting and stopping files
-move partial content handling into ManagedStream
-add delayed stop test
2019-05-06 18:46:26 -04:00
Jack Robison b134e0c9c9 fix blob_get 2019-05-06 18:46:26 -04:00
Jack Robison f8c0e80cfc add __slots__ to BlobInfo, StreamDescriptor, and KademliaPeer 2019-05-06 18:46:26 -04:00
Jack Robison 1f266ebdad add logging for streams we already have 2019-05-06 18:46:26 -04:00
Jack Robison b09e1f8825 fix written_bytes never being 0 2019-05-06 18:46:26 -04:00
Jack Robison c496278a5b Merge pull request #2092 from lbryio/fix_announce_on_storage
Set should_announce=1 to head and descriptor blobs when saving a stream to database
2019-05-06 18:45:36 -04:00
Victor Shyba cacc65d45e fix edge case from test for single blob stream 2019-05-06 18:18:12 -03:00
Victor Shyba 2247b354b3 integration test for announcing both on publish and download 2019-05-06 17:27:12 -03:00
Victor Shyba 43111f0497 bump db revision 2019-05-06 17:19:40 -03:00
Victor Shyba 6c142b4477 add migrator for previous bugs 2019-05-06 17:12:04 -03:00
Victor Shyba e8683a24bd fix should_announce being set on stream creation 2019-05-06 16:49:52 -03:00
Victor Shyba a4d17d1705 re-enable tests for announcing on stream creation 2019-05-06 16:49:03 -03:00
Baltazar Gomez a030e13548 fix incorrect media-type of .cbr archives 2019-05-05 20:57:36 -04:00
Jack Robison 32b8601c38 Merge pull request #2083 from lbryio/fix_get_params
make save_file a parameter instead of a flag so it obeys conf
2019-05-05 19:44:29 -04:00
Victor Shyba f5bd28bfdb make save_file a parameter instead of a flag so it obeys conf 2019-05-05 19:46:42 -03:00
hackrush a54a11dca0 Making tests work on OS X 2019-05-05 16:16:00 -04:00
Lex Berezhny 6cbbe23c7c account_list defaults to 0 confirmations for balance 2019-05-04 18:08:33 -04:00
Lex Berezhny d7c2e4e9b3 fixed bug where lbrynet failed to start with single-key accounts in the wallet 2019-05-04 17:56:39 -04:00
Jack Robison d61ddbb950 Merge pull request #2076 from lbryio/download_improv
make ttfb configurable, fix race condition on writers, ban on time scales, remove idle code for getting unstuck on long downloads, fix foreign key error and improve downloader loop checks
2019-05-02 17:18:16 -04:00
Victor Shyba cf7bb6a391 fix tests and stream being deleted on data timeout 2019-05-02 16:41:48 -03:00
Victor Shyba 68d7328918 always close the write handle 2019-05-02 12:20:49 -03:00
Victor Shyba ff2985cc80 close protocol if transport reuse brings responses before requests 2019-05-02 12:13:33 -03:00
Victor Shyba ac69fcbc5f fix possible division by zero 2019-05-02 11:52:09 -03:00
Victor Shyba 609cf42868 cleanup downloader code 2019-05-01 18:04:45 -03:00
Victor Shyba e43b29fcd1 if the current writer is closed, its fine to open a new one 2019-05-01 15:09:11 -03:00
Victor Shyba 6594b7c50c test concurrency through races on getting a writer 2019-05-01 15:08:47 -03:00
Victor Shyba 971f2a1cef log download speed only if downloading full streams 2019-05-01 14:31:11 -03:00
Victor Shyba fc88261cc6 add head_blob_only to ttfb 2019-05-01 13:59:45 -03:00
Victor Shyba 895b3992c9 set type for stall download timeout 2019-05-01 13:26:51 -03:00
Victor Shyba 41951d8ec2 improve ttfb script confs 2019-05-01 13:17:13 -03:00
Victor Shyba ae2121c5c4 fix tests and new test 2019-05-01 13:16:45 -03:00
Victor Shyba 771fe935f4 test get-delete-get respects existing files 2019-05-01 13:16:32 -03:00
Victor Shyba 60f7766cf7 small adjustment to new_peer_or_finished 2019-05-01 02:55:51 -03:00
Victor Shyba 63bd983012 pylint 2019-05-01 00:42:51 -03:00
Victor Shyba 7ffce1040c fix foreign key error 2019-04-30 19:51:19 -03:00
Victor Shyba ac6835fa60 dont get stuck 2019-04-30 19:51:02 -03:00
Victor Shyba 3cf5c536c0 improve ban timing and downloader loop friction 2019-04-30 17:56:29 -03:00
Lex Berezhny 85cc1a4763 v0.37.0rc5 2019-04-29 17:22:47 -04:00
Lex Berezhny 180cfcfb72 fix chris45 tests 2019-04-29 16:01:59 -04:00
Lex Berezhny 864c1dd901 fix test_claim_commands integration tests 2019-04-29 16:01:59 -04:00
Lex Berezhny aeaa7ba986 switched from sha256 for unencrypted file hash to sha384 2019-04-29 16:01:59 -04:00
Lex Berezhny e7beb6b1a7 fix for adding new account from private_key_string 2019-04-29 14:14:31 -04:00
Lex Berezhny b8325c1f89 regenerated docs 2019-04-29 09:49:39 -04:00
Lex Berezhny a36936b5ff v0.37.0rc4 2019-04-29 00:27:53 -04:00
Victor Shyba 7cf245a0e9 encode timestamp and confirmations into file list 2019-04-29 00:26:14 -04:00
Victor Shyba 9f83cce574 test confirmations and timestamp on file list, being updated on resolve 2019-04-29 00:26:14 -04:00
Victor Shyba 30b113d64a depth -> confirmations 2019-04-29 00:26:14 -04:00
Victor Shyba ccd2023d37 add timestamp field to resolve 2019-04-29 00:26:14 -04:00
Victor Shyba 3127cd11cf test for timestamp field on resolve 2019-04-29 00:26:14 -04:00
Victor Shyba 6e483cba8d tests: add confirm parameters, like the other calls 2019-04-28 22:53:43 -04:00
Victor Shyba 46c03c0cb2 refactor: move generate to blockchain call 2019-04-28 22:53:43 -04:00
Victor Shyba c2ca0a9467 tests: replace sleep with the proper generate call 2019-04-28 22:53:43 -04:00
Victor Shyba b350e74aca fix stream update callback to update the main dict 2019-04-28 22:53:43 -04:00
Victor Shyba de9473f75a test resolve updates file list 2019-04-28 22:53:43 -04:00
Lex Berezhny bfc107fdb3 removed blockchain.block.get_block from wallet server API 2019-04-27 18:03:35 -04:00
Lex Berezhny 6a2efa54ab dropped block_show command from API, users should run a full node if they need access to blocks 2019-04-27 18:03:35 -04:00
Lex Berezhny 72f0e1ca14 added timestamp field to transaction output results from JSON RPC calls 2019-04-27 17:10:49 -04:00
Lex Berezhny 9a1262d557 regenerate docs 2019-04-27 13:13:20 -04:00
Lex Berezhny 706a8e52dc v0.37.0rc3 2019-04-27 13:12:27 -04:00
Lex Berezhny f3c8776878 fixed --preview mode on stream_create/update 2019-04-27 13:11:32 -04:00
Lex Berezhny 539ea04039 fixed signing streams using --channel_name argument 2019-04-27 11:52:29 -04:00
Lex Berezhny 8f96b07559 updating stream claim with new file now correctly updates file list removing old file 2019-04-27 10:55:20 -04:00
Lex Berezhny 0a1e97fe23 cli test fix 2019-04-26 23:37:10 -04:00
Lex Berezhny da637b9c0e added --replace option to stream_update and channel_update, publish defaults to --replace 2019-04-26 23:37:10 -04:00
Lex Berezhny 3618de1227 added --clear_fee ability to stream update command 2019-04-26 17:52:29 -04:00
Victor Shyba 3cb8a63bf9 add channel_name parameter on claim search 2019-04-26 17:30:21 -04:00
Victor Shyba bad99de5b3 add tests for channel_name, which should work for urls too 2019-04-26 17:30:21 -04:00
Lex Berezhny 3d1efbc9bb v0.37.0rc2 2019-04-25 20:40:07 -04:00
Victor Shyba 7bdef3146b fix ttfb 2019-04-25 20:38:33 -04:00
Victor Shyba 2145bfcabf install uvloop on wallet server docker image and default to it 2019-04-25 20:07:10 -04:00
Lex Berezhny b88832d0a9 regenerated docs/api.json 2019-04-24 23:03:30 -04:00
Lex Berezhny db08882f20 v0.37.0rc1 2019-04-24 21:35:59 -04:00
Jack Robison 6f52d36b22 Merge pull request #2020 from lbryio/download-range-requests
Support HTTP 206 partial content requests for streaming downloads
2019-04-24 14:21:41 -04:00
Jack Robison 056a629e87 remove unused property 2019-04-24 13:39:06 -04:00
Jack Robison 1ffb99f9f9 rename streaming_only setting -> save_files 2019-04-24 12:44:12 -04:00
Jack Robison a8e612773d test 2019-04-24 09:48:04 -04:00
Jack Robison 67088b2ca2 test_blob_writers_concurrency 2019-04-24 09:48:04 -04:00
Jack Robison dab49e812f set writer result or error before calling close_handle 2019-04-24 09:48:04 -04:00
Jack Robison 0f3f38d8a3 pylint 2019-04-24 09:48:04 -04:00
Jack Robison 7ba433c2e5 fix setting blob verified event and canceling blob writers
-raise OSError if multiple writers for one peer are opened for a blob
2019-04-24 09:48:04 -04:00
Jack Robison bb7649e8ad test file_save stopped before finished writing with streaming_only on/off 2019-04-24 09:48:04 -04:00
Jack Robison 832537a5cf set deleted downloads as streaming mode on startup 2019-04-24 09:48:04 -04:00
Jack Robison 3234d70270 add peer address/port to hash blob writer 2019-04-24 09:48:04 -04:00
Jack Robison 432fe444f8 fix cache_concurrent decorator 2019-04-24 09:48:04 -04:00
Jack Robison 861c25716e add unit tests for cache_concurrent decorator 2019-04-24 09:48:04 -04:00
Victor Shyba 9ba6653475 raise proper error when sd is not available 2019-04-24 09:48:04 -04:00
Jack Robison 8107e97304 remove debugging /streams endpoint 2019-04-24 09:48:04 -04:00
Jack Robison e2663f6d57 remove unnecessary except/raise 2019-04-24 09:48:04 -04:00
Jack Robison 35479dcbd2 add file_save integration tests 2019-04-24 09:48:04 -04:00
Jack Robison 3a997277aa reduce the amount of Blob objects created 2019-04-24 09:48:04 -04:00
Jack Robison a7d94f4514 add file_save 2019-04-24 09:48:04 -04:00
Jack Robison a0e3338177 only re-reflect streams if the sd blob is readable 2019-04-24 09:48:04 -04:00
Jack Robison d5a353066c missing await 2019-04-24 09:48:04 -04:00
Jack Robison bc29fc4465 type 2019-04-24 09:48:04 -04:00
Jack Robison 987f175fcc logging 2019-04-24 09:48:04 -04:00
Jack Robison 5b1398b588 rename test_streaming.py 2019-04-24 09:48:04 -04:00
Jack Robison 873b58efc9 update example_daemon_settings.yml 2019-04-24 09:48:04 -04:00
Jack Robison 01a81fceed more streaming_only + save_blobs integration testing 2019-04-24 09:48:04 -04:00
Jack Robison a14b0aa573 tests 2019-04-24 09:48:04 -04:00
Jack Robison a1abb7c8fb test range requests 2019-04-24 09:48:04 -04:00
Jack Robison 46b166952b non async blob_completed callback, tests 2019-04-24 09:48:04 -04:00
Jack Robison 1375153fda fix publish updates not being saved to the db, don't query the db for the stream hash 2019-04-24 09:48:04 -04:00
Jack Robison 1a14157d37 fix unawaited task 2019-04-24 09:48:04 -04:00
Jack Robison 40b9fa38b0 update mock dht 2019-04-24 09:48:04 -04:00
Jack Robison 4d0e9e0428 relative http redirect from /get -> /stream 2019-04-24 09:48:04 -04:00
Jack Robison 3a53ef5690 catch TransportNotConnected 2019-04-24 09:48:04 -04:00
Jack Robison 33a68b5cef handle dht is_closing 2019-04-24 09:48:04 -04:00
Jack Robison 63bf084a6a cancel dht response future 2019-04-24 09:48:04 -04:00
Jack Robison a882f2e8c5 merge conflicts 2019-04-24 09:48:04 -04:00
Jack Robison 119e0ec408 cancelled error 2019-04-24 09:48:04 -04:00
Jack Robison 7d782d516a track and close blob readers 2019-04-24 09:48:04 -04:00
Jack Robison d3eac48035 tests, add __slots__ to AbstractBlob 2019-04-24 09:48:04 -04:00
Jack Robison 60c9ae64b4 endpoint to call get and redirect to the partial content stream 2019-04-24 09:48:04 -04:00
Jack Robison 2d314dce60 add save_file argument to get 2019-04-24 09:48:04 -04:00
Jack Robison 287b89db66 fix tests 2019-04-24 09:48:04 -04:00
Jack Robison 7a160f7335 fix tests 2019-04-24 09:48:04 -04:00
Jack Robison 04ddee7d89 blob tests 2019-04-24 09:48:04 -04:00
Jack Robison 3a916a8e8e tests 2019-04-24 09:48:04 -04:00
Jack Robison f125468ebf fix claim not being saved to the db 2019-04-24 09:48:04 -04:00
Jack Robison f0e17cff9a support streaming downloads / range requests 2019-04-24 09:48:04 -04:00
Jack Robison 93267efe0b remove StreamAssembler, refactor ManagedStream and StreamDownloader (WIP) 2019-04-24 09:48:04 -04:00
Jack Robison c663e5a3cf add cache_concurrent decorator 2019-04-24 09:48:04 -04:00
Jack Robison 676f0015aa refactor BlobFile into AbstractBlob, BlobFile, and BlobBuffer classes 2019-04-24 09:48:04 -04:00
Jack Robison d44a79ada2 cache resolve_host, add async_timed_cache decorator 2019-04-24 09:48:04 -04:00
Jack Robison d7ed8ffaa6 add streaming_only and save_blobs settings 2019-04-24 09:48:04 -04:00
Jack Robison b230d8fbcb rename BlobFileManager -> BlobManager 2019-04-24 09:48:04 -04:00
Lex Berezhny 30edd53f3e 0.5.2 2019-04-24 00:11:14 -04:00
Lex Berezhny 7d2428fd17 regenerate api.json 2019-04-23 23:49:39 -04:00
Lex Berezhny 4157045a41 block_show command example 2019-04-23 23:49:39 -04:00
Lex Berezhny 58987857cf v0.36.0 2019-04-22 18:47:58 -04:00
Lex Berezhny fa8858325c add lbry:// to permanent_url response field 2019-04-22 18:41:11 -04:00
Lex Berezhny 3357a419ab ability to overide the file name, hash and size when updating a stream claim 2019-04-22 18:05:49 -04:00
Lex Berezhny a782adea75 regenerate api doc 2019-04-22 14:06:42 -04:00
Lex Berezhny e8991e6d52 improve account locking/unlocing CLI docs 2019-04-22 14:06:42 -04:00
Lex Berezhny 13bd81c2b6 v0.36.0rc4 2019-04-21 23:11:23 -04:00
Victor Shyba 21241b98a4 workaround for having new encoder fields on resolve so app can use them 2019-04-21 22:51:48 -04:00
Lex Berezhny bf467e06c5 added create stream image detection test 2019-04-21 22:12:02 -04:00
Lex Berezhny 886ed6415a added examples which include all metadata passed to create stream and create channel 2019-04-21 21:34:13 -04:00
Lex Berezhny 97db241109 v0.36.0rc3 2019-04-21 20:20:53 -04:00
Lex Berezhny 216379918b renamed txo sub_type to value_type in txo json encoder output 2019-04-21 17:40:40 -04:00
Lex Berezhny fcab4eb9bd make sure to use python3 when running set_build.py 2019-04-21 16:31:46 -04:00
Lex Berezhny 19f300cc45 added protobuf to get and file_list commands 2019-04-21 14:28:43 -04:00
Lex Berezhny 26fec098e4 fix stream unit tests 2019-04-21 14:28:43 -04:00
Lex Berezhny 7824de655e updated stream manager to pull protobuf instead of hex from resolve response 2019-04-21 14:28:43 -04:00
Lex Berezhny 6ef1b1dd59 resolve command also returns protobuf 2019-04-21 14:28:43 -04:00
Lex Berezhny 9fae8f5153 added --include_protobuf to add hexlified protobuf to RPC responses 2019-04-21 14:28:43 -04:00
Lex Berezhny f660691e78 v0.36.0rc2 2019-04-21 00:50:27 -04:00
Lex Berezhny bf60683ba6 update to protobuf change where ClaimList defaults to collection list type 2019-04-21 00:45:51 -04:00
Lex Berezhny 68215699f7 fix chri45 test 2019-04-21 00:45:51 -04:00
Lex Berezhny 1644ffbf22 regenerated docs 2019-04-21 00:45:51 -04:00
Lex Berezhny 71f5061848 exposed new protobuf fields in APIs and updated tests 2019-04-21 00:45:51 -04:00
Lex Berezhny 61cadb5cbe v0.36.0rc1 2019-04-20 02:34:27 -04:00
Lex Berezhny 2034104cfc test fixes 2019-04-20 02:33:41 -04:00
Lex Berezhny 633f725634 stream manager test fix 2019-04-20 02:33:41 -04:00
Lex Berezhny 8bf2e41c95 --contact_email -> --email and --homepage_url -> --website_url 2019-04-20 02:33:41 -04:00
Lex Berezhny fa9a4a75c6 update to an even newer protobufs 2019-04-20 02:33:41 -04:00
Lex Berezhny 90bef98bc3 use TXORef.hash instead of TXORef.id when signing claim with channel 2019-04-20 02:33:41 -04:00
Lex Berezhny 35c43647aa use TXORef.hash instead of TXORef.id when signing claim with channel 2019-04-20 02:33:41 -04:00
Lex Berezhny 449a583286 default to claimType == 1 when it is missing for old claim protobufs 2019-04-20 02:33:41 -04:00
Lex Berezhny b1f813782a regenerated protobuf python code 2019-04-20 02:33:41 -04:00
Lex Berezhny 60080cbd19 updating to new protobufs 2019-04-20 02:33:41 -04:00
Victor Shyba 665dba62c7 simplify save from resolve, deduplicate items 2019-04-20 01:19:40 -04:00
Victor Shyba 76dea9e89b batch save supports 2019-04-20 01:19:40 -04:00
Victor Shyba ebc9ed01c5 make lru_cache a power of two, following py docs 2019-04-20 01:19:40 -04:00
Victor Shyba 55116763e9 cache transactions 2019-04-20 01:19:40 -04:00
Victor Shyba e1a6b4e3d6 fix dict handling mistake 2019-04-20 01:19:40 -04:00
Victor Shyba 911ca8c37e cache and share txs fetches while validating concurrently on batches 2019-04-20 01:19:40 -04:00
Lex Berezhny fe446862c2 txo.hash added 2019-04-19 08:42:12 -04:00
jessop 7f5deaf6c8 adds api examples to docs 2019-04-16 14:18:23 -04:00
binaryflesh 6788e09ae9 PEP-257 docstrings
Signed-off-by: binaryflesh <logan.campos123@gmail.com>
2019-04-16 06:24:08 -04:00
Lex Berezhny 0200756218 put generated commands in groups 2019-04-12 14:24:16 -04:00
ポール ウェッブ a63442f843 Fixes and updated gitignore 2019-04-12 11:53:37 -04:00
Oleg Silkin 74b3da79aa Adds integration tests 2019-04-07 15:27:59 -04:00
Oleg Silkin b6a4589da5 Adds comment group to lbrynet with commands list and create 2019-04-07 15:27:59 -04:00
Oleg Silkin fbbb0a9931 Adds server URL to configuration settings 2019-04-07 15:27:59 -04:00
Oleg Silkin a94d65d522 Adds utility functions for making batch jsonrpc requests to comment server 2019-04-07 15:27:59 -04:00
Oleg Silkin fa2ead002e Fixes incompatible attrs issue when running pip install -e . 2019-04-06 18:43:34 -04:00
Thomas Zarebczan 8dfee4867a Update python3 > python, add mac instructions 2019-04-06 17:10:32 -04:00
Thomas Zarebczan 6614ab8e79 updated with new venv command 2019-04-06 17:10:32 -04:00
Thomas Zarebczan d32c1dfbb3 update install for windowz
Didn't need to install any build tools
Also, venv commands are a bit different.
2019-04-06 17:10:32 -04:00
Lex Berezhny 9f42b958fa run examples fix 2019-04-06 17:08:15 -04:00
Lex Berezhny 574a912f7d run examples as part of ci 2019-04-06 17:08:15 -04:00
Lex Berezhny 2cfcc3cd45 test fix 2019-04-06 17:08:15 -04:00
Lex Berezhny 9af625e627 test fixes 2019-04-06 17:08:15 -04:00
Lex Berezhny 1871c94003 new examples 2019-04-06 17:08:15 -04:00
Lex Berezhny 93220c3874 daemon json rpc return type docs 2019-04-06 17:08:15 -04:00
Lex Berezhny 362ed3f1c8 remove unused import 2019-04-06 17:08:15 -04:00
Lex Berezhny 62a182b8bb automated example generator 2019-04-06 17:08:15 -04:00
Lex Berezhny 6f76d9e24a moved integration testcase.py into lbrynet so that it can be used from scripts 2019-04-06 17:08:15 -04:00
Lex Berezhny 55e83fe05c regenerate api.json 2019-04-06 17:08:15 -04:00
Lex Berezhny 2f0c7d6a3f added title to examples 2019-04-06 17:08:15 -04:00
Lex Berezhny 59e1df9efa generate api examples 2019-04-06 17:08:15 -04:00
Lex Berezhny f0c2d16749 changed account list to use is_default instead of is_default_account and improved blockchain deamon port settings 2019-04-06 15:27:58 -04:00
Victor Shyba fd56dc68cd fix depth on resolve, add tests 2019-04-05 09:40:26 -04:00
Victor Shyba a0a810edc3 make permanent url name#claim_id on torba too 2019-04-05 09:40:26 -04:00
Victor Shyba c1d3b5984d fix old hash reverse confusion 2019-04-05 09:40:26 -04:00
Victor Shyba 1fb719dd61 fix valid_at_height not being shown for name resolutions, which fixes tests 2019-04-05 09:40:26 -04:00
Victor Shyba b87316fb02 make permanent url name#id only for now 2019-04-05 09:40:26 -04:00
Victor Shyba 65f3d78084 fix invalid signatures for missing certificates 2019-04-05 09:40:26 -04:00
Victor Shyba 47bc1526f7 test invalid signature to be False, not accepting None 2019-04-05 09:40:26 -04:00
Victor Shyba 3acabec4bb fixes from review 2019-04-03 00:51:05 -04:00
Victor Shyba 36a62d3756 port orchstr8 fix to our test case as its lbrynet specific 2019-04-03 00:51:05 -04:00
Victor Shyba 2e22473ce3 pylint and smaller fixes 2019-04-03 00:51:05 -04:00
Victor Shyba 0b0fd64642 further refactor resolve.py 2019-04-03 00:51:05 -04:00
Victor Shyba ca17d8597d move resolve from ledger to resolve.py 2019-04-03 00:51:05 -04:00
Victor Shyba d16e2716c8 handle invalid data 2019-04-03 00:51:05 -04:00
Victor Shyba 89c80cedfd test resolve for invalid data 2019-04-03 00:51:05 -04:00
Thomas Zarebczan 021e30ffde fix: extension check
disregard the first period, otherwise the `filename.` test fails.
2019-04-01 22:56:40 -04:00
Thomas Zarebczan eb2cd8fec1 stream type mapping
Mapping extensions to stream types. I went with document / model / binary for the ones we were missing. This can be adjusted later on if not correct, but at least we'll have the main ones. Added extensions for lbry, cbz, cbr also.
2019-04-01 22:56:40 -04:00
Thomas Zarebczan 6356818557 fix: time to first byte script
Fix with for metadata
2019-04-01 16:14:53 -04:00
YULIUS KURNIAWAN KRISTIANTO 8bf724a78b Update CONTRIBUTING.md 2019-04-01 16:12:49 -04:00
YULIUS KURNIAWAN KRISTIANTO 65d0a924c1 Update README.md 2019-04-01 16:12:49 -04:00
Lex Berezhny ef7bd00f22 round USD fees to nearest penny 2019-03-31 18:14:13 -04:00
Lex Berezhny 3c1fdb5114 v0.35.0 2019-03-29 21:51:05 -04:00
Lex Berezhny 83411acfbd split claim_abandon into stream_abandon and channel_abandon 2019-03-29 21:48:55 -04:00
Lex Berezhny 4230812f82 added integration test to make sure publishing the same stream re-signs it with the original channel 2019-03-29 21:14:20 -04:00
Lex Berezhny 7b57ba96dd fix docopt parsing and stream manager tests 2019-03-29 21:14:20 -04:00
Lex Berezhny 668840921f typo fix 2019-03-29 21:14:20 -04:00
Lex Berezhny a3bd335f7f --languages, --locations fully support all attributes, add BTC 2019-03-29 21:14:20 -04:00
Lex Berezhny 4e6bca1389 support_abandon sends new --keep support to same address as the first support being abandoned 2019-03-29 21:14:20 -04:00
Lex Berezhny c4b15920ff dont show supports to foreign address if the inputs are also foreign 2019-03-29 21:14:20 -04:00
Lex Berezhny 2a55e583e7 added to docstring that a channel_id is a channel claim id 2019-03-29 21:14:20 -04:00
Lex Berezhny 2c4fde06c5 --new-signing-key -> --new_signing_key 2019-03-29 21:14:20 -04:00
Lex Berezhny eca83d7f47 --clear-tags -> --clear_tags 2019-03-29 21:14:20 -04:00
Lex Berezhny 9b21bd1a6f 0.5.1 2019-03-29 21:11:13 -04:00
Lex Berezhny 169ef00b9e + txo.pubkey_hash property 2019-03-29 17:59:18 -04:00
Lex Berezhny 87e818ac39 v0.35.0rc2 2019-03-27 22:20:09 -04:00
Lex Berezhny adab7bd7b1 fix file command tests 2019-03-27 21:57:22 -04:00
Lex Berezhny 750c0e4f4f fix cli test 2019-03-27 21:57:22 -04:00
Lex Berezhny 5f8482bdb7 hash -> sd_hash, encoding fixes, update to new protobufs 2019-03-27 21:57:22 -04:00
Lex Berezhny cd266477fb stop regenerating v1 protobufs 2019-03-27 21:57:22 -04:00
Lex Berezhny 7860b956ff fixes and tests for publish command 2019-03-27 21:57:22 -04:00
Thomas Zarebczan 148481a5ca doh 2019-03-27 21:54:19 -04:00
Thomas Zarebczan 93faa276a5 update BIP32 2019-03-27 21:54:19 -04:00
Victor Shyba 53e67a6e81 fixes from rebase 2019-03-27 10:31:37 -04:00
Victor Shyba 3ad6808d87 remove unused claim_id_hash 2019-03-27 10:31:37 -04:00
Victor Shyba 071ab32ab7 fix hash references 2019-03-27 10:31:37 -04:00
Victor Shyba ab87588250 use torba parsing for addresses too, erase old code 2019-03-27 10:31:37 -04:00
Victor Shyba 22db29ee1d use torba tx parsing on wallet server 2019-03-27 10:31:37 -04:00
Lex Berezhny 66bf6c61f0 v0.35.0rc1 2019-03-25 23:57:59 -04:00
Lex Berezhny 9abf40d21f stream_type fix 2019-03-25 23:45:10 -04:00
Lex Berezhny ea4444edbe fix internal transaction api test 2019-03-25 22:54:08 -04:00
Lex Berezhny 51bb8fd283 pylint 2019-03-25 22:54:08 -04:00
Lex Berezhny 183e3ac1c7 claim search 2019-03-25 22:54:08 -04:00
Lex Berezhny 8087b457c6 reorganized commands and finished support_abandon with --keep 2019-03-25 22:54:08 -04:00
Lex Berezhny fedb7c23d0 bring back blocks_behind in wallet status 2019-03-25 22:54:08 -04:00
Lex Berezhny e2964f9728 test cli fix, take 2 2019-03-25 22:54:08 -04:00
Lex Berezhny e0cea60c87 fix cli test 2019-03-25 22:54:08 -04:00
Lex Berezhny c708c1db07 audio/video/image stream types 2019-03-25 22:54:08 -04:00
Lex Berezhny 1cb696fb5c fix test_claim_show 2019-03-25 22:54:08 -04:00
Lex Berezhny 7cd0213eb9 fixed chris45 again 2019-03-25 22:54:08 -04:00
Lex Berezhny 2007c48af3 fixed tests for setting claim values 2019-03-25 22:54:08 -04:00
Lex Berezhny 50721786d2 skip test_claim_list for now 2019-03-25 22:54:08 -04:00
Lex Berezhny 6d7eb259cb test normalization fixes 2019-03-25 22:54:08 -04:00
Lex Berezhny cc5852a9b9 update pyyaml due to security alert 2019-03-25 22:54:08 -04:00
Lex Berezhny 0592cee567 claim integration test fixes 2019-03-25 22:54:08 -04:00
Lex Berezhny 814eaca524 cli tests fixed 2019-03-25 22:54:08 -04:00
Lex Berezhny e06916f5ae fixing unit tests 2019-03-25 22:54:08 -04:00
Victor Shyba d61e1e9d8c fix proto v1 integration test 2019-03-25 22:54:08 -04:00
Lex Berezhny 25436893bc pylint and a few other fixes 2019-03-25 22:54:08 -04:00
Lex Berezhny 5b9d41fd5c regeneate script after modifying protobufs 2019-03-25 22:54:08 -04:00
Victor Shyba b1720d1659 fix setting sd hash 2019-03-25 22:54:08 -04:00
Lex Berezhny 4b3a6ab875 big support tests passes 2019-03-25 22:54:08 -04:00
Lex Berezhny 67dcede2d8 lots of stuff deleted 2019-03-25 22:54:08 -04:00
Victor Shyba 0872a6fea3 partial test_claim_list_by_channel 2019-03-25 22:54:08 -04:00
Victor Shyba a3aed3219f fix test_abandoned_channel_with_signed_claims 2019-03-25 22:54:08 -04:00
Lex Berezhny af73e1a2f5 chris45 passing again 2019-03-25 22:54:08 -04:00
Lex Berezhny 55bbf5aa74 progress on chris45 2019-03-25 22:54:08 -04:00
Lex Berezhny c1fbb02bfe dont validate inputs in json_response_encoder 2019-03-25 22:54:08 -04:00
Lex Berezhny d47575e8e0 progress 2019-03-25 22:54:08 -04:00
Victor Shyba 8fc4f4aa73 cleanup recent test changes 2019-03-25 22:54:08 -04:00
Victor Shyba fe9aa666e9 fix name is not part of old sigs 2019-03-25 22:54:08 -04:00
Victor Shyba a9e0eeba7e add tests and fix verification of old signatures 2019-03-25 22:54:08 -04:00
Victor Shyba 0830917afb delete unused test method 2019-03-25 22:54:08 -04:00
Victor Shyba 1ed086e90f bring back test for picking oldest on dupe claims 2019-03-25 22:54:08 -04:00
Victor Shyba 56db631626 fix signed channel tests, WIP on tampering 2019-03-25 22:54:08 -04:00
Victor Shyba aa029955ee fix tx api tests 2019-03-25 22:54:08 -04:00
Victor Shyba f2236eb7a4 resolve: check if there is a cert before fetching a tx 2019-03-25 22:54:08 -04:00
Lex Berezhny df649341cd json output formatting fix for Claim 2019-03-25 22:54:08 -04:00
Lex Berezhny a3c3143868 fixed remaining references to lbrynet.extras.wallet -> lbrynet.wallet 2019-03-25 22:54:08 -04:00
Lex Berezhny d0201fe7c7 base integration test class should default to VERBOSITY=WARN, not DEBUG 2019-03-25 22:54:08 -04:00
Lex Berezhny 664a2a324d fix import error on stream unit test 2019-03-25 22:54:08 -04:00
Lex Berezhny ea09aca175 pylint fixes and ClaimDict->Claim 2019-03-25 22:54:08 -04:00
Lex Berezhny ca92fddecf pylint 2019-03-25 22:54:08 -04:00
Lex Berezhny 8259d7c680 txo.claim_hash and other fixes related to proto3 refactor 2019-03-25 22:54:08 -04:00
Victor Shyba 6653da2973 all 2019-03-25 22:54:08 -04:00
Lex Berezhny e74998209d for victor 2019-03-25 22:54:08 -04:00
Lex Berezhny 2bea0f697a pylint 2019-03-25 22:54:08 -04:00
Lex Berezhny 1d68bef6f2 client side pretty much done 2019-03-25 22:54:08 -04:00
Lex Berezhny 22bacc907b moved test_channel_signing.py from tests/unit/schema to tests/unit/wallet 2019-03-25 22:54:08 -04:00
Lex Berezhny 1e8ad900c7 renamed test_signing.py to test_channel_signing.py 2019-03-25 22:54:08 -04:00
Lex Berezhny cd2b535afb moved claim signing to TXO 2019-03-25 22:54:08 -04:00
Lex Berezhny 9dfe7bbcf6 imports updated after moving lbrynet.extras.wallet to lbrynet.wallet 2019-03-25 22:54:08 -04:00
Lex Berezhny 120618ffd5 updated imports due to lbrynet.extras.wallet -> lbrynet.wallet move 2019-03-25 22:54:08 -04:00
Lex Berezhny e8abcbb06e moved lbrynet.extras.wallet -> lbrynet.wallet 2019-03-25 22:54:08 -04:00
Lex Berezhny 1ec8f0b0b4 wip 2019-03-25 22:54:08 -04:00
Lex Berezhny 323694fb12 removed address.py 2019-03-25 22:54:08 -04:00
Lex Berezhny d5e37acd8f minor Fee changes 2019-03-25 22:54:08 -04:00
Lex Berezhny 129a886f2b + fee.amount 2019-03-25 22:54:08 -04:00
Lex Berezhny cd15230a92 getting close to a nice model 2019-03-25 22:54:08 -04:00
Lex Berezhny c1b7fdc68e deleted bunch of stuff 2019-03-25 22:54:08 -04:00
Lex Berezhny 9fd67aaad6 removed legacy_schema_v1 2019-03-25 22:54:08 -04:00
Lex Berezhny ce9dc1836e moved schema/current_schema/claim.py into schema/claim.py 2019-03-25 22:54:08 -04:00
Lex Berezhny 2e557d3a36 converted everyting in schema/legacy/* into schema/jsonschema.py 2019-03-25 22:54:08 -04:00
Lex Berezhny 5df05cb37e make schema.types a module 2019-03-25 22:54:08 -04:00
Lex Berezhny 326b62dc57 moved proto2/proto3 to types/v1 and types/v2 respectively 2019-03-25 22:54:08 -04:00
Lex Berezhny 25cc7ba284 updated schema to latest types: Claim uses oneof type for Channel/Stream 2019-03-25 22:54:08 -04:00
Victor Shyba 550f064302 baseschema 2019-03-25 22:54:08 -04:00
Victor Shyba 11086b93f1 migrate other fields 2019-03-25 22:54:08 -04:00
Victor Shyba df5662136d migrate cert + adapt to latest types repo 2019-03-25 22:54:08 -04:00
Victor Shyba 291d65070d move common schema values to constants.py 2019-03-25 22:54:08 -04:00
Victor Shyba 9b64ee33dc fix imports: schema.schema -> schema.legacy_schema_v1 2019-03-25 22:54:08 -04:00
Victor Shyba 5404d26f2f schema.schema -> schema.legacy_schema_v1 2019-03-25 22:54:08 -04:00
Victor Shyba 26ade1f228 bump merged proto3 2019-03-25 22:54:08 -04:00
Victor Shyba b49f1f1b54 fix imports 2019-03-25 22:54:08 -04:00
Victor Shyba 3bafa2ad22 regenerate protos 2019-03-25 22:54:08 -04:00
Victor Shyba 378ec44f3c update make to generate proto2 and proto3 2019-03-25 22:54:08 -04:00
Victor Shyba 9d0d17b422 rename proto proto2 2019-03-25 22:54:08 -04:00
Lex Berezhny aa93b94b61 fix unit tests 2019-03-25 22:27:52 -04:00
Lex Berezhny d7dadc2f0e + __not_like db condition 2019-03-25 22:04:48 -04:00
Lex Berezhny 33ac9eea13 fix is_valid_address 2019-03-24 21:53:52 -04:00
Lex Berezhny 9ff41f748e upgrades 2019-03-24 17:09:55 -04:00
Lex Berezhny 9589ac6d02 include tx in db get_txos() 2019-03-22 03:03:48 -04:00
Lex Berezhny cfa74985e0 Revert "add hash property to txoref"
This reverts commit 38ae660a68.
2019-03-20 14:17:14 -04:00
Victor Shyba 38ae660a68 add hash property to txoref 2019-03-20 14:02:14 -04:00
Lex Berezhny ca4a07f766 ledger.is_valid_address added 2019-03-20 01:36:30 -04:00
Victor Shyba f7c8145283 fix travis not to build wallet on PR 2019-03-19 12:22:00 -04:00
Lex Berezhny 9281bdb61b lazy parse scripts 2019-03-18 23:25:00 -04:00
Lex Berezhny 80b65ae7e7 OP_0 can be an OP but also an empty string, figure this out contextually in parser 2019-03-18 18:13:49 -04:00
Lex Berezhny 0de17730ca v0.34.0 2019-03-15 16:21:29 -04:00
Jack Robison 58743ba19b Merge pull request #1987 from lbryio/more-dht-tests
improve dht unit tests
2019-03-15 15:38:51 -04:00
Jack Robison f2fefbe287 add lbrynet.dht.blob_announcer unit tests 2019-03-15 14:29:02 -04:00
Jack Robison 60a24f0e6e add lbrynet.dht.protocol.data_store unit tests 2019-03-15 12:44:55 -04:00
Jack Robison 664f91bfab add lbrynet.dht.protocol.distance unit tests 2019-03-15 12:44:41 -04:00
Jack Robison 44f92a271f move dht tests to reflect the real directory structure 2019-03-15 12:01:21 -04:00
Jack Robison 6565ca8558 improve lbrynet.dht.serialization unit tests 2019-03-14 18:45:56 -04:00
Jack Robison 3b0ba3e534 v0.34.0rc2 2019-03-14 16:03:39 -04:00
Jack Robison 8106eb1fa5 Merge pull request #1986 from lbryio/fix-timeout-no-peers
fix default download timeout not being set
2019-03-14 15:47:12 -04:00
Jack Robison c9b2829227 fix/test fixed_peer_delay and added_fixed_peers 2019-03-14 15:08:26 -04:00
Jack Robison 4f2f8b16f3 fix pending downloads not being removed 2019-03-14 13:52:00 -04:00
Jack Robison 2ab2b846b1 fix default download timeout not being set 2019-03-14 13:07:11 -04:00
Jack Robison 2df7284c29 Merge pull request #1985 from lbryio/fix-ttfb-analytics
fix the time to the head blob including the time to the sd blob in analytics
2019-03-14 11:17:37 -04:00
Jack Robison ab5d71ff61 fix the time to the head blob including the time to the sd blob in analytics 2019-03-13 16:24:23 -04:00
Lex Berezhny 99b289f135 v0.34.0rc1 2019-03-12 20:29:10 -04:00
Lex Berezhny 6db02e4362 show build type setting when building 2019-03-12 20:23:10 -04:00
Lex Berezhny 98d8b7d463 sync uses new apply APIs in torba 2019-03-12 20:22:41 -04:00
Lex Berezhny 96f1883fc4 check modified_on 2019-03-12 20:22:41 -04:00
Lex Berezhny a914de155a working sync 2019-03-12 20:22:41 -04:00
Lex Berezhny 2dee0ff0bf wip account manifest command 2019-03-12 20:22:41 -04:00
Jack Robison 993cb43e5b fix tests
-fix cleanup of a failed download
2019-03-12 20:22:01 -04:00
Jack Robison 5d212a0f82 time to first bytes analytics 2019-03-12 20:22:01 -04:00
Lex Berezhny 61a254f580 0.4.0 2019-03-12 15:36:24 -04:00
Jack Robison 0cc602ca53 Merge pull request #1978 from lbryio/sigint
handle sigint
2019-03-12 15:06:08 -04:00
Lex Berezhny 4ba90a727b make apply not abstract 2019-03-12 15:05:07 -04:00
Lex Berezhny 27558ffd89 implement abstract method for pylint 2019-03-12 15:05:07 -04:00
Lex Berezhny c44de48481 + account.apply and addressmanager.apply 2019-03-12 15:05:07 -04:00
Victor Shyba f390ff144d handle sigint 2019-03-12 13:43:12 -04:00
Jack Robison b266cde511 Merge pull request #1980 from lbryio/batched-channel-name-initialization
use _batched_select to populate channel names when initializing files
2019-03-12 13:41:58 -04:00
Jack Robison d0a1541d0a fix test 2019-03-12 13:00:43 -04:00
Jack Robison 4cbde1b08f use _batched_select to populate channel names when initializing files 2019-03-12 09:58:19 -04:00
Lex Berezhny a021ddeffd + account.modified_on 2019-03-11 22:50:39 -04:00
Victor Shyba cdd25e3b0f pylint 2019-03-11 22:50:39 -04:00
Victor Shyba 1286417aec apply scrypt as kdf, add tests 2019-03-11 22:50:39 -04:00
Lex Berezhny fcac4f1488 fix encrypt 2019-03-11 22:50:39 -04:00
Lex Berezhny f655fefe52 typing fix 2019-03-11 22:50:39 -04:00
Lex Berezhny 7979f0a683 typing fix 2019-03-11 22:50:39 -04:00
Lex Berezhny d9df2ead84 pylint fixes 2019-03-11 22:50:39 -04:00
Lex Berezhny b07ca20e0e working sync 2019-03-11 22:50:39 -04:00
Lex Berezhny c6740d8d70 v0.33.0 2019-03-08 20:00:45 -05:00
Lex Berezhny 46358142ed v0.33.0rc1 2019-03-06 22:18:04 -05:00
Lex Berezhny f0f651fc77 rely on torba cffi explicit version 2019-03-05 09:32:43 -05:00
Lex Berezhny 5d428138ed try older cffi on windows build 2019-03-05 09:32:43 -05:00
Brannon King 8d9da61c7d reverted temporary code for full local stack from 2019-03-05 09:32:43 -05:00
Brannon King c326a9382c changed to pull down the newer lbrycrd 2019-03-05 09:32:43 -05:00
Brannon King 5bdd87c904 removed name-to-claim lookup table
added unit test for Greek and made it work


pylint: revert bad move


revert claim_sequence changes


fixed broken test
2019-03-05 09:32:43 -05:00
Jack Robison ad134fe8bb v0.32.4 2019-03-01 17:24:55 -05:00
Jack Robison 2f3c616acd v0.32.4rc1 2019-03-01 16:33:59 -05:00
Jack Robison 3caa2a5039 Merge pull request #1968 from lbryio/analytics
restore old download analytics
2019-03-01 16:08:18 -05:00
Jack Robison e38a0f7a95 logging 2019-03-01 15:26:45 -05:00
Jack Robison 9de3f3ff09 don't block on sending download analytics events 2019-03-01 15:22:19 -05:00
Jack Robison df8934d472 restore old download analytics and update tests 2019-03-01 14:50:06 -05:00
Jack Robison be4a96c1b3 Merge pull request #1965 from lbryio/certifi
fix ssl errors, use certifi to make ssl contexts
2019-02-28 13:14:17 -05:00
Jack Robison d97d978961 use certifi for all operating systems 2019-02-28 12:45:56 -05:00
Jack Robison f52e44b56a typing 2019-02-28 12:42:23 -05:00
Jack Robison 10d73fe480 use certifi 2019-02-28 12:40:11 -05:00
Jack Robison 4fd2d994f2 Merge pull request #1964 from lbryio/update-aiohttp
update aiohttp and pylint
2019-02-28 12:38:17 -05:00
Jack Robison 3a1cd8985c Merge pull request #70 from lbryio/update-aiohttp
update aiohttp and astroid requirements
2019-02-28 12:17:08 -05:00
Jack Robison d95903d219 update aiohttp and pylint 2019-02-28 12:04:07 -05:00
Jack Robison a79be0513b pylint 2019-02-28 12:00:44 -05:00
Jack Robison 569ac502ed update aiohttp and astroid requirement
pylint 2.3.0 fixes aiohttp failure
2019-02-28 11:56:39 -05:00
Jack Robison 591e66bfbb Merge pull request #1952 from lbryio/no_trap
stop trapping CancelledError
2019-02-27 11:06:02 -05:00
Victor Shyba e047748961 log dropped api request instead of stack trace 2019-02-26 17:42:45 -05:00
Victor Shyba 9f071bbe36 pylint 2019-02-26 17:42:45 -05:00
Victor Shyba 2d8ebe25ed refactor so we can stop trapping CancelledError everywhere 2019-02-26 17:42:45 -05:00
Victor Shyba 7a96e742f2 stop trapping CancelledError 2019-02-26 17:42:45 -05:00
Jack Robison 13c7026b20 Merge pull request #1961 from lbryio/pin-requirements
Pin requirements
2019-02-26 17:42:13 -05:00
Jack Robison 876a3c68dd pin requirements 2019-02-26 17:20:16 -05:00
Lex Berezhny b86d2e6e73 use older astroid for pylint 2019-02-26 15:46:47 -05:00
Lex Berezhny f8d24d3402 older aiohttp but newer astroid 2019-02-26 15:43:20 -05:00
Lex Berezhny 589768864a try latest mypy and aiohttp 2019-02-26 15:38:37 -05:00
Lex Berezhny bd2512c27e pin dependency versions 2019-02-26 15:33:14 -05:00
Jack Robison 573617d6b6 Merge pull request #1957 from lbryio/dht_memory
add LRU to KademliaPeer creation
2019-02-26 10:10:33 -05:00
Victor Shyba e8c9abb944 add LRU to KademliaPeer creation 2019-02-26 09:42:32 -05:00
Jack Robison 8cee627dfa Merge pull request #1959 from lbryio/pin-jsonschema
pin jsonschema version to 2.6.0
2019-02-26 09:41:20 -05:00
Jack Robison ab4a5cee01 pin jsonschema version 2019-02-26 09:23:41 -05:00
Lex Berezhny 5bb0a07fae pip with git does not appear to work in wine 2019-02-22 20:52:42 -05:00
Lex Berezhny 5ca594b9fd upgrade pip in wine 2019-02-22 20:52:42 -05:00
Lex Berezhny 3910964df1 plyvel no longer installed unless platform is explicitely linux in torba/setup.py 2019-02-22 20:52:42 -05:00
Lex Berezhny 9e4c1120fa 0.3.2 2019-02-22 19:26:03 -05:00
Lex Berezhny 4eff0d2660 only install plyvel on linux 2019-02-22 19:25:27 -05:00
Jack Robison faa1935203 Merge pull request #1946 from lbryio/file-type-fix
lowercase/stripped comparison for known types
2019-02-22 13:07:09 -05:00
Jack Robison ba518e2a0e update test 2019-02-22 11:12:31 -05:00
Thomas Zarebczan 2a82ff7cc5 Update mime_types.py 2019-02-22 11:11:10 -05:00
Jack Robison 673b958020 v0.32.3 2019-02-21 19:38:14 -05:00
Jack Robison 3f4ed0a896 Merge pull request #1951 from lbryio/fix_invalid_blob_decode_startup
Fix invalid blob decode startup
2019-02-21 19:36:02 -05:00
Victor Shyba f49578cdbc fix integration test 2019-02-21 21:14:51 -03:00
Victor Shyba b834f29634 nullify length on blob delete 2019-02-21 21:00:28 -03:00
Victor Shyba 61132312dd test nullify length on blob delete 2019-02-21 21:00:20 -03:00
Victor Shyba fcdd9295b4 delete upon failing to decode JSON from sd blob 2019-02-21 20:53:21 -03:00
Victor Shyba edb5083a5e test that corrupted sd blobs gets delete if fails to decode 2019-02-21 20:53:04 -03:00
Victor Shyba 4e7d88311f raise proper exception on undecode-able blobs 2019-02-21 20:42:51 -03:00
Victor Shyba aa40337ded test that invalid blobs raise proper exception 2019-02-21 20:42:36 -03:00
Jack Robison 690e2e7e0c v0.32.3rc1 2019-02-21 18:37:19 -05:00
Jack Robison 0c67469321 Merge pull request #1950 from lbryio/fix-slow-ping-queue-shutdown
Fix slow ping queue shutdown
2019-02-21 18:12:22 -05:00
Jack Robison fc64d8d1a8 fix slow ping queue shutdown 2019-02-21 17:44:02 -05:00
Jack Robison 7fe9513857 v0.32.2 2019-02-21 14:46:36 -05:00
Jack Robison e76107c318 v0.32.2rc3 2019-02-21 14:28:13 -05:00
Jack Robison 3868dea819 don't join buckets if there is only one 2019-02-21 14:18:48 -05:00
Jack Robison c52541197f Merge pull request #1949 from lbryio/fix-join-buckets
fix cases where buckets could be unjoined after removing a peer
2019-02-21 14:08:34 -05:00
Jack Robison 03cd6b7200 fix cases where buckets could be unjoined after removing a peer 2019-02-21 13:53:35 -05:00
Jack Robison 6915a7a553 v0.32.2rc2 2019-02-21 13:16:32 -05:00
Jack Robison 7ad29b4bfb Merge pull request #1948 from lbryio/duplicates-in-routing
Test and fix edge cases for adding peers to routing
2019-02-21 13:11:34 -05:00
Jack Robison 4294bf742d simplify/fix ping queue 2019-02-21 13:00:23 -05:00
Jack Robison 57ad9f1c52 test_add_peer_after_handle_request 2019-02-21 11:34:25 -05:00
Jack Robison e260de59f1 fix updating/setting contact triple 2019-02-21 10:50:11 -05:00
Jack Robison 154702e6dd fix KBucket.add_peer 2019-02-21 10:50:11 -05:00
Jack Robison d06d998c81 improve KBucket test_add_peer 2019-02-21 10:50:11 -05:00
Jack Robison 8aff860bfe Merge pull request #1945 from lbryio/fix_delete_log
Fix delete log
2019-02-21 10:14:29 -05:00
Victor Shyba d7ba90bbea update ttfb script 2019-02-20 17:10:36 -03:00
Victor Shyba 59325ad8c1 fix delete log 2019-02-20 17:04:08 -03:00
Jack Robison 1f72e27272 improve time_to_first_byte.py 2019-02-20 17:04:08 -03:00
Jack Robison aa41b8b558 Merge pull request #1944 from lbryio/fix-seed-node-startup
fix seed node startup
2019-02-20 15:03:47 -05:00
Jack Robison 2496e7ed44 improve time_to_first_byte.py 2019-02-20 14:43:40 -05:00
Jack Robison ee12985135 dont ping outselves 2019-02-20 14:15:31 -05:00
Jack Robison 4c7f357441 v0.32.2rc1 2019-02-20 12:12:04 -05:00
Jack Robison 8944a36291 Merge pull request #1942 from lbryio/split_buckets_under_index
add split_buckets_under_index config setting for seed nodes
2019-02-20 12:00:46 -05:00
Jack Robison cb5d428d12 add split_buckets_under_index config setting for seed nodes 2019-02-20 11:25:50 -05:00
Jack Robison f175e8826c Merge pull request #1939 from lbryio/missing_await
refactor file_name and full_path as it can be None, fix missing await, stop using blob_manager.storage on stream_manager and use your own
2019-02-20 10:44:35 -05:00
Victor Shyba 38ed212c01 refactor, dont call storage from blob_manager + missing await 2019-02-19 21:45:14 -03:00
Victor Shyba d035c13883 file_path can be None, add helper to check if stream output file exists instead 2019-02-19 21:45:14 -03:00
Jack Robison 82ce2ccee6 update docs 2019-02-19 17:27:31 -05:00
Jack Robison 2cf9b2dd9e Merge pull request #1938 from lbryio/fix-kbucket-indexing
fix KBucket.key_in_range
2019-02-19 15:03:16 -05:00
Jack Robison fcbdaa336f fix KBucket.key_in_range
The range_min and range_max of a KBucket where mistakenly bounding the node id of a peer to be added rather than the distance from the local node id to the peer node id
2019-02-19 13:57:11 -05:00
Lex Berezhny 3ae6137edb v0.32.1 2019-02-18 22:56:35 -05:00
Lex Berezhny 0fdd2ae178 added tx to JSON response from get 2019-02-18 22:53:11 -05:00
Victor Shyba 8c5007fe07 assert amount transferred to correct account 2019-02-18 22:53:11 -05:00
Victor Shyba d2150cf142 fix balance check 2019-02-18 22:53:11 -05:00
Victor Shyba ce2bb22929 add tests for paid downloads 2019-02-18 22:53:11 -05:00
Lex Berezhny 1ce3adb6a0 reduce integration test verbosity 2019-02-18 21:45:37 -05:00
Lex Berezhny c2dc1f3c1d v0.32.1rc1 2019-02-18 17:17:20 -05:00
Victor Shyba b36c22e2f4 unban after 10 seconds, give up after 60 2019-02-18 17:16:10 -05:00
Victor Shyba 71b66ab337 test stall unban 2019-02-18 17:16:10 -05:00
Lex Berezhny 461c3c8408 downloading header from S3 correctly tracks progress 2019-02-18 17:15:40 -05:00
Thomas Zarebczan 03396c1993 remove invalid reflector_component test config 2019-02-18 17:10:42 -05:00
Jack Robison fa7e9923f4 Merge pull request #1932 from lbryio/fix-1930
Fix blob_list for sd_hash including stream terminator
2019-02-18 14:41:00 -05:00
Jack Robison 3f7e466c97 return failure message for invalid blob hash in blob_delete 2019-02-18 12:18:56 -05:00
Jack Robison 4483380352 fix terminator blob hash (None) being included when running blob_list for an sd hash 2019-02-18 12:12:14 -05:00
Lex Berezhny 4a0a6b0243 v0.32.0 2019-02-16 20:17:41 -05:00
Thomas Zarebczan a003e794ba listen on internal port instead of upnp redirect 2019-02-16 18:55:58 -05:00
Lex Berezhny 051fabc180 v0.32.0rc3 2019-02-16 15:07:07 -05:00
Jack Robison ca686041e7 Merge pull request #1925 from lbryio/fix-extension
fix application/x-ext mime types
2019-02-15 21:11:05 -05:00
Jack Robison fc450eb164 add TestMimeTypes, pylint 2019-02-15 20:42:55 -05:00
Thomas Zarebczan 652d7cbd24 consider . extension 2019-02-15 20:40:55 -05:00
Thomas Zarebczan 43b372c0fe fix extension - remove dot
The rest of our extensions don't have a dot. Also, strip off any characters and lowercase it to be safe.
2019-02-15 20:40:55 -05:00
Jack Robison eaacf7e034 Merge pull request #1926 from lbryio/fix_unfair_ban
Fix p2p ignoring good peers
2019-02-15 20:39:49 -05:00
Victor Shyba 576da1bc30 do not keep trying when blob is being written 2019-02-15 21:20:06 -03:00
Victor Shyba 5c217f1544 resovle reflector IP before adding it 2019-02-15 21:20:06 -03:00
Jack Robison cfbf0dbe0f Merge pull request #1919 from lbryio/sync-blobs-on-startup
Sync blobs on startup
2019-02-15 18:40:33 -05:00
Jack Robison fd081496c0 fix blocking on data being written for resumed streams 2019-02-15 17:35:46 -05:00
Jack Robison 32b4405a4c logging
fix https://github.com/lbryio/lbry/issues/1881
2019-02-15 17:26:28 -05:00
Jack Robison 0d558bd552 fix default file list sorting 2019-02-15 17:26:24 -05:00
Jack Robison ca835f3f80 fix 1883 2019-02-15 16:26:13 -05:00
Jack Robison dbb6ba6241 recover streams with missing sd blobs, handle previous sd blob bugs
-test download and recover stream with old key sorting
2019-02-15 15:21:29 -05:00
Jack Robison a228d20137 move store_stream and store_file to standalone functions 2019-02-15 15:21:07 -05:00
Jack Robison f1b60e3ef2 fix get_blobs_for_stream 2019-02-15 15:21:07 -05:00
Jack Robison 39737c790f sync blob files in database with those in directory on startup 2019-02-15 15:21:07 -05:00
Jack Robison dba4a09d38 Merge pull request #1922 from lbryio/supress_invalid_token_error_to_debug
supress invalid token error to debug
2019-02-15 15:19:58 -05:00
Victor Shyba 918122e7cf supress invalid token error to debug 2019-02-15 16:31:47 -03:00
Victor Shyba a49f479a5f move wait to where its needed on assembler 2019-02-15 10:26:05 -05:00
Victor Shyba 320aabbd70 delete duplicated method accident from rebase 2019-02-15 10:26:05 -05:00
Jack Robison 19fdcad5af Merge pull request #1905 from lbryio/faster-file-delete
Faster file delete
2019-02-14 17:23:55 -05:00
Jack Robison 1734e08297 fix test 2019-02-14 16:34:37 -05:00
Jack Robison 9d89f9b834 use executemany instead of batched_operation 2019-02-14 15:57:24 -05:00
Jack Robison 56c41b2fea fix integration test
-cancel running reflector uploads from publishes upon shutdown
-refactor blob.delete to be non-async
-delete blobs synchronously
2019-02-14 15:43:26 -05:00
Jack Robison 9c8593d88e fix file_delete being slow 2019-02-14 12:33:48 -05:00
Jack Robison 85f41887fd don't create BlobFile object when deleting if it doesn't already exist 2019-02-14 12:28:42 -05:00
Mark Michon 8eede6dfc7 Update LICENSE
Updating license to 2019
2019-02-14 08:47:26 -05:00
Victor Shyba 0d8c37a63b resolve uri->url on ttfb 2019-02-14 00:30:24 -05:00
Lex Berezhny d2ba3fdcfb v0.32.0rc2 2019-02-14 00:18:21 -05:00
Victor Shyba be5ab0f307 test get retry 2019-02-14 00:10:44 -05:00
Victor Shyba 04f70d1f31 stop when data timeout 2019-02-14 00:10:44 -05:00
Victor Shyba 60ba07cc17 add tests for incomplete files, update existing one 2019-02-14 00:10:44 -05:00
Victor Shyba dbc8cfe5ae delete incomplete files on assembler stop 2019-02-14 00:10:44 -05:00
Victor Shyba d91e8f5996 revert import to original one + remove print 2019-02-13 23:33:49 -05:00
Victor Shyba b6df2b9d28 missing None for optional field + pylint 2019-02-13 23:33:49 -05:00
Victor Shyba 5c1ab42277 resume should put the stream into wait as done usually on get calls 2019-02-13 23:33:49 -05:00
Victor Shyba bbdc96e762 file_name as a property of managed stream, to reflect internal changes 2019-02-13 23:33:49 -05:00
Victor Shyba fc0c949187 properly update file name on downloader/assembler 2019-02-13 23:33:49 -05:00
Victor Shyba 3fce8e4023 tests resuming downloads with conflicting file names 2019-02-13 23:33:49 -05:00
Victor Shyba 37bb765a2e resume downloads if DHT disabled 2019-02-13 23:33:49 -05:00
Jack Robison 09726876ad pylint 2019-02-13 22:50:43 -05:00
Jack Robison 45c6ad3f6d remove commits after pragma 2019-02-13 22:50:43 -05:00
Jack Robison 97e4d51c29 tests and feedback 2019-02-13 22:50:43 -05:00
Jack Robison f134bc264b add run_nofk to AIOSQLite to run without foreign keys 2019-02-13 22:50:43 -05:00
hackrush 21ca4f60f7 Stop the daemon on calling lbrynet stop 2019-02-13 14:22:23 -05:00
Victor Shyba 119d498bce apply a timeout on test wait so if it hangs it fails fast 2019-02-13 13:10:27 -05:00
Victor Shyba 8108aa39c1 verify all blobs got transferred 2019-02-13 13:10:27 -05:00
Victor Shyba 0a068011d8 fix chunked response parsing 2019-02-13 13:10:27 -05:00
Victor Shyba 1193d2cea4 adds tests for chunked responses 2019-02-13 13:10:27 -05:00
Lex Berezhny 76ecb5d50d renamed resolve argument url to urls 2019-02-13 11:55:12 -05:00
Lex Berezhny dbc1164562 v0.32.0rc1 2019-02-12 21:35:41 -05:00
Victor Shyba 57eb56b92d timeout earlier on connect 2019-02-12 21:02:25 -05:00
Victor Shyba 6c68ba1d22 add timeout->retry tests] 2019-02-12 21:02:25 -05:00
Victor Shyba e2170ec83d fix download timeout types 2019-02-12 20:58:31 -05:00
Victor Shyba 266e777655 add integration test for different download errors 2019-02-12 20:58:31 -05:00
Lex Berezhny 68905d82b9 pass conf instead of data_dir to migrate_db, fixes #1898 2019-02-12 18:10:34 -05:00
Lex Berezhny 396c49f7c6 passing get, file list and file delete tests 2019-02-12 11:54:00 -05:00
Lex Berezhny 5e509c9fd6 testing get 2019-02-12 11:54:00 -05:00
Lex Berezhny e6b6a3f55e dedicated transaction commands tests 2019-02-12 11:54:00 -05:00
Lex Berezhny 019af119d9 moved test_cli.py up one level 2019-02-12 11:54:00 -05:00
Lex Berezhny 58da2f16c2 use absolute imports instead of relative 2019-02-12 11:54:00 -05:00
Lex Berezhny 8aa50aa14f cleaned up resolve command 2019-02-12 11:54:00 -05:00
Lex Berezhny 65c6c83b51 reusable command testcase 2019-02-12 11:54:00 -05:00
Lex Berezhny f497989871 split out claim commands 2019-02-12 11:54:00 -05:00
Lex Berezhny 5366779b1b moved all other tests out of test_chris45.py and into dedicated files 2019-02-12 11:54:00 -05:00
Lex Berezhny a3f59f041a dedicated integration tests for account commands 2019-02-12 11:54:00 -05:00
Lex Berezhny d560b0c1cd dedicated integration tests for resolve command 2019-02-12 11:54:00 -05:00
Lex Berezhny e94229121a explicitly set LEDGER 2019-02-12 11:54:00 -05:00
Lex Berezhny 097291a498 moved wallet integration tests up one level to simplify 2019-02-12 11:54:00 -05:00
Jack Robison afdc44ca13 Merge pull request #1894 from lbryio/fix-max-key-fee
fix max_key_fee not being followed
2019-02-11 16:27:29 -05:00
Brannon King cabc58feb4 fix decoding error in torba log 2019-02-11 16:11:01 -05:00
Jack Robison 6732b302bb logging 2019-02-11 15:31:31 -05:00
Jack Robison c9d54bb049 fix max_key_fee 2019-02-11 15:26:13 -05:00
Alex Grintsvayg f65e70df2f better resolve description 2019-02-11 12:34:57 -05:00
Lex Berezhny 364f95612a 0.3.0 2019-02-10 23:11:53 -05:00
Jack Robison 4541e754dd Merge pull request #1892 from lbryio/batch-query-files
Improve startup time
2019-02-08 20:43:52 -05:00
Jack Robison 9f89c62e8a download full streams in time_to_first_byte.py 2019-02-08 20:14:10 -05:00
Jack Robison 6386815a29 fix reflect_streams not being followed 2019-02-08 20:13:26 -05:00
Jack Robison 0c93090d2c faster is_valid_hashcharacter 2019-02-08 19:57:26 -05:00
Jack Robison 45404b6ad4 faster is_valid_blobhash 2019-02-08 19:54:59 -05:00
Jack Robison aeb2891baa faster population of channel names 2019-02-08 19:51:13 -05:00
Jack Robison 8abfdc82b2 faster blob manager startup 2019-02-08 17:37:35 -05:00
Jack Robison da9b4e317a limit blobs to announce 2019-02-08 16:29:55 -05:00
Jack Robison e414cc5c48 whitespace 2019-02-08 16:11:56 -05:00
Jack Robison 939edb2f18 logging
-call loop.set_debug when using --verbose
2019-02-08 16:06:38 -05:00
Jack Robison 868110a6f2 populate stream manager with a single batch query 2019-02-08 16:06:35 -05:00
Jack Robison 869a8b712b Merge pull request #1889 from lbryio/download_stuc
Refactor peer scoring and maintain connections to fix stuck downloads, remove locking from ping queue
2019-02-08 15:10:13 -05:00
Victor Shyba 9ecec52412 check lock acquired to prevent races 2019-02-08 04:39:28 -03:00
Victor Shyba 8b25acff6e do not wait blob to be written, log errors from races 2019-02-08 04:32:53 -03:00
Victor Shyba e2b06677b5 defer db insert during downloads 2019-02-08 03:45:23 -03:00
Victor Shyba af1619ebfb disable race limiting for now 2019-02-08 03:32:38 -03:00
Victor Shyba 3352e0e4f4 score peers by speed 2019-02-08 03:08:08 -03:00
Victor Shyba 141d68a2cd fix write race condition 2019-02-08 02:39:29 -03:00
Victor Shyba c06ec6cd69 manage connections, reusing them when possible 2019-02-08 02:27:58 -03:00
Victor Shyba 1be5dce30e stop racing during long streams 2019-02-08 01:28:03 -03:00
Victor Shyba 73da223ae1 conf: give up first contact earlier, but let the transfer go longer 2019-02-08 01:05:58 -03:00
Victor Shyba 2d7eb83518 change score calculation, wait for active peers too, simplify peer sorting/keeping 2019-02-08 01:04:38 -03:00
Victor Shyba ad03f91d24 let cancelled errors flow 2019-02-08 01:03:13 -03:00
Victor Shyba 5586a226c2 bypass parser during download 2019-02-07 21:19:31 -03:00
Victor Shyba a616582733 wait on connection tasks 2019-02-07 20:11:57 -03:00
Victor Shyba 43ac928f0b remove dht locks 2019-02-07 20:11:57 -03:00
Lex Berezhny a121110743 v0.31.0 2019-02-06 14:28:40 -05:00
Jack Robison 8bc5bb4e14 Merge pull request #1873 from lbryio/fix_incomplete_downloads_assembly
test incomplete streams, respect real length, delete on incomplete as…
2019-02-06 14:26:40 -05:00
Victor Shyba 3a15ff4bcd test incomplete streams, respect real length, delete on incomplete assembly 2019-02-06 15:56:06 -03:00
Jack Robison 3508da4993 Merge pull request #1866 from lbryio/non-async-close-blob
refactor blob.close to be non-async, speed up deleting blobs and streams
2019-02-06 13:52:37 -05:00
Jack Robison dfdc8eda00 improve test, fix stream_finished_event 2019-02-06 13:32:50 -05:00
Jack Robison 58f6cb71c6 test stream manager 2019-02-06 13:08:57 -05:00
Jack Robison ae11c5bb4b fix setting finished event 2019-02-06 12:18:24 -05:00
Jack Robison f22b5da170 fix raised error for a peer address mismatch 2019-02-06 12:11:22 -05:00
Jack Robison c1c6d5bc99 fix deleting partial downloads when stopped and previous streams when updating a publish 2019-02-06 11:43:05 -05:00
Jack Robison 0e972ec2ae refactor BlobFile.close to be non async 2019-02-06 11:00:57 -05:00
Jack Robison f8dc0f298e Merge pull request #1868 from lbryio/fix_error_response
format error dict on get
2019-02-06 10:37:31 -05:00
Victor Shyba 8b5803c945 format error dict on get 2019-02-06 11:55:12 -03:00
Lex Berezhny 1314f75219 release script shows PRs that did not get included in changelog 2019-02-06 09:51:09 -05:00
Jack Robison be90ec51e4 release-text 2019-02-06 09:00:59 -05:00
Jack Robison e06c903731 Merge pull request #1867 from lbryio/stream-descriptor-validation
Add stream descriptor tests
2019-02-05 21:43:45 -05:00
Jack Robison 76c8447a56 add stream descriptor tests 2019-02-05 21:25:31 -05:00
Jack Robison 82b53e108c Merge pull request #1865 from lbryio/fix-delete-previous-stream
Fix deleting incomplete downloads when they are stopped
2019-02-05 15:43:55 -05:00
Jack Robison 7c8ad0fe30 fix deleting incomplete downloads upon stop
fix deleting previous streams (not assembled/published files) for a claim being updated
2019-02-05 15:33:08 -05:00
Jack Robison 073ffdd049 proto and type args 2019-02-05 13:36:25 -05:00
Jack Robison 7fe211d134 fix resolve_host 2019-02-05 13:31:57 -05:00
Jack Robison 43d424978d Merge pull request #1862 from lbryio/fix_announce
adds announce tests on stream creation/assemble, refactor calls to set announce
2019-02-05 12:58:56 -05:00
Victor Shyba a4ced4dbd2 set verified after callback 2019-02-05 13:17:00 -03:00
Victor Shyba c6b910a5f3 use time_getter on storage 2019-02-05 13:14:59 -03:00
Victor Shyba 63d1ec3c28 fixes from jack comments 2019-02-05 13:01:12 -03:00
Victor Shyba c31e61586d tests: fix value, compare to blobs len 2019-02-05 13:01:12 -03:00
Victor Shyba 7b8f42e6d0 remove dead set_should_announce code 2019-02-05 13:01:12 -03:00
Victor Shyba 2c275efa48 remove redundant calls to should announce 2019-02-05 13:01:12 -03:00
Victor Shyba 30846f932b test that creating a stream marks sd and head to announce 2019-02-05 13:01:12 -03:00
Victor Shyba 6ead932ccb announces naturally, removes redundant calls 2019-02-05 13:01:12 -03:00
Victor Shyba e9b58577dd tests: verify completed and to announce after assembling streams 2019-02-05 13:01:12 -03:00
Victor Shyba f766dbefe0 announce and set completed after inserts 2019-02-05 13:01:12 -03:00
Jack Robison f8a7d2ebe2 add release_texts to release.py 2019-02-05 09:58:42 -05:00
Jack Robison af43b21ef6 Merge pull request #1860 from lbryio/fix-announce
Fix setting blob completed and next announce time
2019-02-04 18:39:10 -05:00
Jack Robison 0685bc5689 include torba in log file 2019-02-04 18:25:47 -05:00
Jack Robison 8cda3143f2 fix setting should_announce and blob status 2019-02-04 18:22:44 -05:00
Jack Robison 5af351a234 reflector logging 2019-02-04 17:33:25 -05:00
Jack Robison b9934a7eb7 Merge pull request #1859 from lbryio/fix_invalid_data_client
handle and log if downloading from bad sendfile implementations
2019-02-04 17:19:57 -05:00
Victor Shyba b82d2c3120 handle and log if downloading from bad sendfile implementations 2019-02-04 19:13:31 -03:00
Jack Robison 8e607fcc2a Merge pull request #1858 from lbryio/fix-blob-sendfile
Fix blob sendfile
2019-02-04 17:12:50 -05:00
Jack Robison 2bce7cecfa fix sendfile on windows 2019-02-04 16:48:48 -05:00
Jack Robison e141cc9e22 Merge pull request #1857 from lbryio/handle-invalid-data-and-fix-cancel-blob-request
Handle invalid data and fix cancel blob request
2019-02-04 16:47:35 -05:00
Jack Robison 8883587cc7 logging 2019-02-04 16:19:27 -05:00
Jack Robison c85991704d catch invalid data error 2019-02-04 16:19:27 -05:00
Jack Robison 377442d9f0 fix request_blob cancellation 2019-02-04 16:19:24 -05:00
Jack Robison ad30b7df6d Merge pull request #1855 from lbryio/findvalue-always-include-close-contacts
Always return close peers from findValue
2019-02-04 15:30:17 -05:00
Jack Robison f0b49fc153 always return close peers from findValue 2019-02-04 15:07:42 -05:00
Jack Robison 6033399f05 Merge pull request #1854 from lbryio/no-self-queries
fix trying downloading from oneself, prune querying peer from findValue results
2019-02-04 15:00:13 -05:00
Jack Robison ae1162f5b0 don't try downloading from oneself
-don't return the querying peer in results for findValue
2019-02-04 14:50:53 -05:00
Jack Robison 7d7d4c35c3 Merge pull request #1852 from lbryio/close_protocol_reflector
refactor reflect to close protocol
2019-02-04 14:42:18 -05:00
Victor Shyba ad71a9a89d refactor reflect to close protocol 2019-02-04 13:34:18 -03:00
Jack Robison 13b3e4ba8e v0.31.0rc5 2019-02-04 10:22:43 -05:00
Jack Robison 43799168ba Merge pull request #1850 from lbryio/resume
Properly resume streams and fixup with statement
2019-02-04 10:18:31 -05:00
Victor Shyba 736a949cb5 resume streams on startup 2019-02-03 20:49:09 -03:00
Victor Shyba 11bb1cc7ee use with clause to avoid leaking fds 2019-02-03 20:41:47 -03:00
Jack Robison dbce49e567 Merge pull request #1849 from lbryio/fix-connection-status
Fix connection status
2019-02-03 17:00:14 -05:00
Jack Robison dcd78bd86e always delete file from downloads if download was not complete 2019-02-03 16:43:40 -05:00
Jack Robison e5af041627 fix connection status message 2019-02-03 16:20:52 -05:00
Jack Robison 73f4e954e5 Merge pull request #1848 from lbryio/fix-announce-time
Fix setting blob announce time
2019-02-03 16:18:33 -05:00
Jack Robison 0a027cf2de logging 2019-02-03 15:47:19 -05:00
Jack Robison 3f98b16800 fix blob announce time, use time.time instead of loop.time 2019-02-03 15:47:15 -05:00
Jack Robison bbadece75d remove unused packages from extension-pkg-whitelist in pylintrc 2019-02-03 15:02:43 -05:00
Jack Robison 8f5a8ab8ba fix peer_ping 2019-02-02 02:02:19 -05:00
Jack Robison 91e581422c v0.31.0rc4 2019-02-02 00:30:57 -05:00
Jack Robison b54d7185a1 Merge pull request #1847 from lbryio/re-reflect
add re-reflect task to stream manager, add concurrent_reflector_uploads to config
2019-02-01 23:11:06 -05:00
Jack Robison f2968aab22 add re-reflect task to stream manager, add concurrent_reflector_uploads to config 2019-02-01 22:59:41 -05:00
Jack Robison 4e27eaf8e6 Merge pull request #1846 from lbryio/create-directories-if-dont-exist
create data. download, and wallet directories if they don't exist on startup
2019-02-01 22:57:27 -05:00
Jack Robison ccbe8aa0eb create data. download, and wallet directories if they don't exist on startup 2019-02-01 22:45:28 -05:00
Jack Robison 478f79353a fix bytes error 2019-02-01 22:20:11 -05:00
Jack Robison c3b0151943 fix dht status error during startup 2019-02-01 22:12:07 -05:00
Jack Robison ba818354af Merge pull request #1844 from lbryio/reassemble-when-deleted-from-downloads
Reassemble when deleted from downloads
2019-02-01 21:59:36 -05:00
Jack Robison e96b75a0d0 fix file/download paths
move download-from-uri logic into stream manager
2019-02-01 21:40:35 -05:00
Jack Robison efe4afd09e fix blob announce time 2019-02-01 21:28:59 -05:00
Jack Robison c7cb6822cc logging 2019-02-01 21:28:59 -05:00
Jack Robison c75665d3f0 update file_list docs, add blobs_remaining to file list args and to file dict result 2019-02-01 21:28:59 -05:00
Jack Robison 744375b2c0 re-assemble file / resume downloads 2019-02-01 21:28:59 -05:00
Jack Robison c1b4a012ec Merge pull request #1845 from lbryio/stuck_peers_fixup
fix active peers not being removed after closing + parsing of data stream
2019-02-01 21:28:20 -05:00
Victor Shyba 40da5b6cb6 rework data parser 2019-02-01 20:09:14 -03:00
Victor Shyba b36cf3e36c remove closing peers from active connections 2019-02-01 20:09:14 -03:00
Jack Robison 4e3b529aec Merge pull request #1840 from lbryio/close-transport-on-sendfile-fail
close blob server transport on sendfile failing
2019-02-01 15:13:07 -05:00
Jack Robison 0a08f445a4 close blob server transport on sendfile failing 2019-02-01 15:11:37 -05:00
Jack Robison 3eac99c509 Merge pull request #1841 from lbryio/general_fixes_downloader
General fixes downloader
2019-02-01 15:10:36 -05:00
Victor Shyba f5bf1463af fixes from review 2019-02-01 17:09:37 -03:00
Victor Shyba ca4a55ef28 set verified event earlier and remove stop awaits 2019-02-01 16:05:14 -03:00
Victor Shyba 832be0758b let cancel propagate 2019-02-01 16:05:14 -03:00
Victor Shyba 25050fdeeb fixes for writers (check inside lock and dont hold verified bytes) 2019-02-01 16:05:14 -03:00
Jack Robison df0635103e Merge pull request #1839 from lbryio/faster-stream-manager-startup
faster stream manager startup
2019-02-01 14:00:19 -05:00
Jack Robison dabd9cbe1e faster stream manager startup 2019-02-01 13:53:03 -05:00
Jack Robison 42d7fbdbb3 Merge pull request #1838 from lbryio/fix-update-claim-missing-stream
Fix publishing an update where we don't have the previous stream
2019-02-01 13:52:34 -05:00
Jack Robison 2336015f9a fix publish
fixes https://github.com/lbryio/lbry/issues/1826
- remove `sources` argument from `publish`
- only add/update files we have
- delete existing stream if updating with a new one
2019-02-01 13:35:18 -05:00
Jack Robison 5a28128b69 Merge pull request #1835 from lbryio/remove-peer-manager-lock
Remove peer manager lock
2019-01-31 21:04:21 -05:00
Jack Robison 31445c7797 remove lock from PeerManager 2019-01-31 20:57:48 -05:00
Jack Robison 16d0ff8376 Merge pull request #1833 from lbryio/fix-publish-update
Fix publish when making an update with an existing stream
2019-01-31 20:56:34 -05:00
Jack Robison 87f5d1bd4a fix updating claim with existing stream 2019-01-31 20:49:19 -05:00
Jack Robison be6e8744b0 Merge pull request #1834 from lbryio/remove-blob-request-lock
remove blob request asyncio.Lock
2019-01-31 20:46:01 -05:00
Jack Robison 2e9acd1baa remove blob request asyncio.Lock 2019-01-31 20:19:50 -05:00
Jack Robison 607ccefe90 Merge pull request #1831 from lbryio/cleanup-downloader
Download and publish fixes
2019-01-31 18:17:26 -05:00
Jack Robison df86937f4e fix deleting a file mid download 2019-01-31 17:59:42 -05:00
Jack Robison 957d0290df add torba logging 2019-01-31 16:53:54 -05:00
Jack Robison 40b90ce79b fix hanging download 2019-01-31 16:53:27 -05:00
Jack Robison aa7f0845cc catch sendfile errors 2019-01-31 16:10:27 -05:00
Jack Robison 6e69d306de tests 2019-01-31 15:39:23 -05:00
Jack Robison d85d3dfaa4 increase default max_connections_per_download 2019-01-31 15:39:20 -05:00
Jack Robison 2b035009ef handle both urls and ip addresses for fixed nodes and reflector servers 2019-01-31 15:39:20 -05:00
Jack Robison f9fd62c214 add fixed peers immediately if the dht has no peers 2019-01-31 15:39:20 -05:00
Jack Robison e8a1953815 add fixed peers immediately if dht is skipped 2019-01-31 15:39:20 -05:00
Jack Robison 34776e7eb9 fix test 2019-01-31 15:39:20 -05:00
Jack Robison a4419f562f pylint 2019-01-31 15:39:20 -05:00
Jack Robison c0160ab4e9 update time_to_first_byte.py 2019-01-31 15:39:20 -05:00
Jack Robison e60e5b3919 fix updating content claim for a file
-fix file name and download dir not being hex encoded during publish, add repair script
2019-01-31 15:39:20 -05:00
Jack Robison 3589cc9977 call update_reflected_stream upon reflector finishing 2019-01-31 15:39:20 -05:00
Jack Robison 2e978c00b2 prioritize peers to download from 2019-01-31 15:39:20 -05:00
Jack Robison 2335243de9 pylint 2019-01-31 15:39:20 -05:00
Jack Robison 6ad68eabd9 tests 2019-01-31 15:39:20 -05:00
Jack Robison ca5c638124 tests 2019-01-31 15:39:20 -05:00
Jack Robison 262b9a624b update file_reflect 2019-01-31 15:39:20 -05:00
Jack Robison 05915c540a set fully reflected if the server indicates it had all of the stream 2019-01-31 15:39:20 -05:00
Jack Robison 6aef6a80b7 refactor downloader
split peer accumulation out, use Queues instead of locks
2019-01-31 15:39:20 -05:00
Jack Robison 16efe9ba95 add fixed_peer_delay setting 2019-01-31 15:39:20 -05:00
Jack Robison fcda9500fe add accumulate_peers to Node
simplifies collecting peers during a download by giving a queue of hashes to search for and a queue or peers as they are found
2019-01-31 15:39:20 -05:00
Alex Grin 64dec821b7 Update README.md 2019-01-31 14:24:44 -05:00
Jack Robison d5f6e58aad Update README.md 2019-01-31 14:15:38 -05:00
Jack Robison 087b65522e Merge pull request #1830 from lbryio/dht_fixes
Dht fixes
2019-01-31 13:52:00 -05:00
Victor Shyba 5ee4b2173a fix probes bugs, partial DHT refactor 2019-01-31 01:49:52 -03:00
Victor Shyba 3e1c3d1e4b use node_id parameter instead of your own 2019-01-31 01:11:09 -03:00
Victor Shyba 21f2b2dcd7 cast timestamp to integer before querying 2019-01-31 01:09:57 -03:00
Jack Robison ffc7fc0793 fix uncaught exception in BlobServerProtocol when client disconnects during a blob transfer 2019-01-30 12:03:05 -05:00
Jack Robison 15dbe5e54e Merge pull request #1828 from lbryio/fix_chunked
fix chunked requests for server
2019-01-30 10:25:30 -05:00
Victor Shyba 2e633c0ee6 fix chunked requests for server 2019-01-30 12:11:05 -03:00
Jack Robison 933d58d49b logging 2019-01-29 20:47:02 -05:00
Victor Shyba 45669dad33 simplify add_keys 2019-01-29 13:17:35 -05:00
Victor Shyba 19e326c140 simplify generate_keys 2019-01-29 13:17:35 -05:00
Victor Shyba e1c0caa357 add executemany and translate batches to it 2019-01-29 13:17:35 -05:00
Jack Robison 8f7cf0b38f verbose logging 2019-01-29 13:08:11 -05:00
Jack Robison adf378b6e9 Merge pull request #1825 from lbryio/fix-hanging-download
Fix hanging download
2019-01-28 23:44:42 -05:00
Jack Robison 1a4f083e76 skip test 2019-01-28 22:50:06 -05:00
Jack Robison 7bde09dcf8 add download_blob_from_peer.py 2019-01-28 22:41:16 -05:00
Jack Robison 4630fd38aa fix async generator junction tests 2019-01-28 22:39:36 -05:00
Jack Robison 418def56ac fix log 2019-01-28 22:38:17 -05:00
Jack Robison cefe3eb520 fix file_delete 2019-01-28 22:01:43 -05:00
Jack Robison 64dffa306f logging, cleanup 2019-01-28 22:01:43 -05:00
Jack Robison 7d33b4f1f3 fix unawaited task 2019-01-28 22:01:43 -05:00
Jack Robison f507d95198 always add fixed peers if configured after 2s 2019-01-28 22:01:43 -05:00
Jack Robison 2a7d80f8b4 logging 2019-01-28 22:01:43 -05:00
Jack Robison 305034ffb2 Merge pull request #1824 from lbryio/fix-ping-timeout
fix timeout error
2019-01-28 21:53:32 -05:00
Jack Robison bfaebf0085 fix timeout error 2019-01-28 17:20:14 -05:00
Jack Robison a94cf04e87 Merge pull request #1822 from lbryio/update-settings
Update settings
2019-01-28 17:17:56 -05:00
Jack Robison 71f9f8ae9c fix race condition in reflector server 2019-01-28 16:51:11 -05:00
Jack Robison 330862e487 fix https://github.com/lbryio/lbry/issues/1297 2019-01-28 16:00:37 -05:00
Jack Robison 778d3826ab clean up settings and use them 2019-01-28 15:51:27 -05:00
Jack Robison 4a11cf007f remove unused settings and constants 2019-01-27 19:51:44 -05:00
Jack Robison f6e7700c4c Merge pull request #1818 from lbryio/fix-find-value
fix error encoding find value response
2019-01-26 17:26:23 -05:00
Jack Robison 573285a5bb fix error encoding find value response 2019-01-26 17:04:11 -05:00
Lex Berezhny 60ed4e503f refactored settings_set to support all settings 2019-01-26 00:04:36 -05:00
Lex Berezhny aa1405887e use float instead of decimal for now 2019-01-25 23:20:43 -05:00
Lex Berezhny cced217e43 update doc format for max_key_fee 2019-01-25 23:20:43 -05:00
Lex Berezhny 8a3b8d2df1 max_key_fee in yaml as null 2019-01-25 23:20:43 -05:00
Lex Berezhny be0bd3bdea refactored --max-key-fee to be more ergonomic 2019-01-25 23:20:43 -05:00
Lex Berezhny e01c73a7f8 pylint 2019-01-25 21:00:20 -05:00
Lex Berezhny 134338db4b dropped help, commands and resolve_name commands 2019-01-25 21:00:20 -05:00
Lex Berezhny f47a685734 help message for start command 2019-01-25 20:28:47 -05:00
Lex Berezhny 6e29f7e99b user is shown help when typing lbrynet help 2019-01-25 20:28:47 -05:00
Lex Berezhny 026d93e92a Update .travis.yml
use `stage: build` for the docker builds to get some parallelism.
2019-01-25 17:28:06 -05:00
Lex Berezhny a55802c1de v0.31.0rc3 2019-01-25 17:05:38 -05:00
Victor Shyba 9d3e1af69b [wallet server] stop executor after stopping server and wait for shutdown event 2019-01-25 17:02:09 -05:00
Jack Robison 894397a327 Merge pull request #1810 from lbryio/async-reflector
async reflector
2019-01-25 16:41:37 -05:00
Jack Robison 3e3f6c0e22 fix test 2019-01-25 16:20:19 -05:00
Jack Robison a7610e3d34 async reflector 2019-01-25 16:10:17 -05:00
Jack Robison d0e6e74e1b Merge pull request #1809 from lbryio/fix-blob-server
fix blob server protocol factory
2019-01-25 16:07:07 -05:00
Jack Robison 126d71978b disable slow test 2019-01-25 15:38:38 -05:00
Jack Robison 69bcb052de fix blob server protocol factory 2019-01-25 13:10:40 -05:00
Victor Shyba 071af12d1a release wallet server master tag every master merge 2019-01-25 12:20:42 -05:00
Victor Shyba 662f2bff4c [wallet server] take control of executor 2019-01-25 11:32:08 -05:00
Victor Shyba 9a9d37cc50 [wallet server] improved/stabler signal handlers 2019-01-25 11:32:08 -05:00
Victor Shyba 3e4c9f25a8 signal handlers cant call async functions 2019-01-25 11:32:08 -05:00
Lex Berezhny b7d62ca084 v0.31.0rc2 2019-01-25 10:46:20 -05:00
Jack Robison fa97f7bd38 fix written_bytes error 2019-01-25 10:23:03 -05:00
Lex Berezhny 4a35247016 fixes for toggle config, passing servers as list, etc 2019-01-25 10:19:24 -05:00
Lex Berezhny d0526ad7d5 bring --config back to start command 2019-01-25 10:19:24 -05:00
Lex Berezhny 19b529cf38 removed unused import 2019-01-25 10:19:24 -05:00
Lex Berezhny 143353be44 fix cli unit test 2019-01-25 10:19:24 -05:00
Lex Berezhny 0373196707 max_key_fee is plain YAML now 2019-01-25 10:19:24 -05:00
Lex Berezhny 7a601db23f cli fixup 2019-01-25 10:19:24 -05:00
Victor Shyba ed61c7feec add step to build wallet server images on release 2019-01-24 19:20:23 -05:00
Jack Robison 77ece96356 fix config error if $HOME is None
-not seen on travis due to HOME being manually set to /tmp
2019-01-24 12:42:55 -05:00
Lex Berezhny 80440fd6bd v0.31.0rc1 2019-01-24 00:13:55 -05:00
Lex Berezhny ed6537db79 integration test fixes 2019-01-24 00:13:15 -05:00
Lex Berezhny 79e4767f69 unused imports 2019-01-24 00:13:15 -05:00
Lex Berezhny 0ee279bfd7 fix startup/shutdown and file_list --full-status argument 2019-01-24 00:13:15 -05:00
Lex Berezhny 5249d2a3ca removed unused methods 2019-01-24 00:13:15 -05:00
Lex Berezhny 4d5823f473 distro package is in setup.py, does not make sense to wrap it in try/except ModuleNotFoundError 2019-01-24 00:13:15 -05:00
Lex Berezhny c2b9909397 moved tests/unit/lbrynet_daemon/test_docs.py into tests/unit/test_cli.py 2019-01-24 00:13:15 -05:00
Lex Berezhny afff362544 component tests need AsyncioTestCase 2019-01-24 00:13:15 -05:00
Lex Berezhny 64eb7dbc49 renamed test_Component_Manager.py to test_component_manager.py 2019-01-24 00:13:15 -05:00
Lex Berezhny 2dcd187d69 fixed component manager tests 2019-01-24 00:13:15 -05:00
Lex Berezhny 3d9ea92118 remove unnecessary logger configuration for BitcoinRPC 2019-01-24 00:13:15 -05:00
Lex Berezhny 13a18d422e dropped envparse 2019-01-24 00:13:15 -05:00
Lex Berezhny 1bbbbd289c TestCase -> AsyncioTestCase 2019-01-24 00:13:15 -05:00
Lex Berezhny e33b74f60c import mock from unittest m0ar 2019-01-24 00:13:15 -05:00
Lex Berezhny cd76e0c296 import mock from unittest 2019-01-24 00:13:15 -05:00
Lex Berezhny 3b61222e5a pylint 2019-01-24 00:13:15 -05:00
Lex Berezhny 2ca438c36e passing integration tests 2019-01-24 00:13:15 -05:00
Jack Robison 8948d5be86 don't shut down immediately after startup 2019-01-24 00:13:15 -05:00
Jack Robison a3e64f9cef remove dht requirement from stream manager component 2019-01-24 00:13:15 -05:00
Jack Robison 09afec1961 import 2019-01-24 00:13:15 -05:00
Jack Robison f9c465a636 f string 2019-01-24 00:13:15 -05:00
Jack Robison 89efe3d1b9 minimal changes to cli.py 2019-01-24 00:13:15 -05:00
Jack Robison 94cec1b86d fix 2019-01-24 00:13:15 -05:00
Jack Robison d62ac0b608 KeyboardInterrupt 2019-01-24 00:13:15 -05:00
Jack Robison 5596eac930 merge conflicts 2019-01-24 00:13:15 -05:00
Jack Robison 59f5381b0a update tests 2019-01-24 00:13:15 -05:00
Jack Robison 65ae1b3f08 update setup.py, travis, and wine_build 2019-01-24 00:13:15 -05:00
Jack Robison 977a7b34f4 update pylintrc 2019-01-24 00:13:15 -05:00
Jack Robison c01728a6a7 update and add scripts 2019-01-24 00:13:15 -05:00
Jack Robison 5bb94d744e storage 2019-01-24 00:13:15 -05:00
Jack Robison 494917158c update daemon and components 2019-01-24 00:13:15 -05:00
Jack Robison ea15674e62 delete old dht functional tests 2019-01-24 00:13:15 -05:00
Jack Robison 34a409ade5 delete log_support, looping_call_manager, and compat 2019-01-24 00:13:15 -05:00
Jack Robison 273e614715 delete old reflector code 2019-01-24 00:13:15 -05:00
Jack Robison 34a725f7ad fix conf 2019-01-24 00:13:15 -05:00
Jack Robison bb6112276f delete old tests 2019-01-24 00:13:15 -05:00
Jack Robison db75f0baef update SQLiteStorage 2019-01-24 00:13:15 -05:00
Jack Robison 30689bbf8d update/add config
remove the following settings:
  peer_search_timeout
  sd_download_timeout
  run_reflector_server
  download_mirrors
  pointtrader_server

add the following settings:
  blob_download_timeout
  peer_connect_timeout
  node_rpc_timeout
2019-01-24 00:13:15 -05:00
Jack Robison 4bf7742c19 async lbrynet.stream 2019-01-24 00:13:15 -05:00
Jack Robison 69b259c285 async lbrynet.blob_exchange 2019-01-24 00:13:15 -05:00
Jack Robison 2fa5233796 async lbrynet.dht 2019-01-24 00:13:15 -05:00
Jack Robison a5524d490c async lbrynet.blob 2019-01-24 00:13:15 -05:00
Jack Robison c713fac2d9 lbrynet.error 2019-01-24 00:13:15 -05:00
Lex Berezhny 7a038bbb98 cli bug fixes 2019-01-23 13:25:10 -05:00
Victor Shyba eba88c1df7 fix resolve for undecodeable claims 2019-01-23 13:22:23 -05:00
Victor Shyba 881e5aba68 simplify claim_list 2019-01-23 13:22:23 -05:00
Victor Shyba 368f0683ec fix claim_list for channels 2019-01-23 09:36:27 -05:00
Victor Shyba 94dc8c8520 add tests for claim_list 2019-01-23 09:36:27 -05:00
Lex Berezhny 2cb9d5c636 orchstr service transfer command works without wallet availability 2019-01-16 13:28:12 -05:00
Lex Berezhny a53b60fca4 use correct env variable for setting SPV hostname 2019-01-16 13:05:23 -05:00
Lex Berezhny 09b77972d9 convert localhost to 127.0.0.1 bypassing /etc/hosts which resolves to ::1 and fails on default docker setup 2019-01-15 15:24:08 -05:00
Lex Berezhny 7a8c2fc98b removed hard coded ipv4 and replaced it with being able to provide hostname to --spv 2019-01-15 00:22:55 -05:00
Lex Berezhny 355989d6d1 test_sync.py fixes 2019-01-13 22:59:57 -05:00
Lex Berezhny 08814ad811 test_sync.py fixes 2019-01-13 22:52:49 -05:00
Lex Berezhny 6a00cf9942 mypy fixes 2019-01-13 22:44:46 -05:00
Lex Berezhny e79202de3c support for linking between nodes 2019-01-13 22:38:36 -05:00
Lex Berezhny 527e288568 explicitely specify ipv4 2019-01-13 14:50:24 -05:00
Lex Berezhny ed32d17936 use guarded stop methods in orchstr8/node.py 2019-01-13 11:44:32 -05:00
Lex Berezhny 3a8e1cd4a5 initial push of workbench 2019-01-13 02:59:54 -05:00
Lex Berezhny 186de045b3 pylint 2019-01-10 17:17:12 -05:00
Lex Berezhny feee534829 silence sql transaction errors 2019-01-10 17:03:09 -05:00
Jack Robison f4bf260cdb fix improperly handled CancelledError on shutdown 2019-01-09 12:15:14 -05:00
Lex Berezhny 442138ef36 added AdvanceTimeTestCase 2019-01-07 03:02:15 -05:00
Lex Berezhny b9b411ec30 instal older aiohttp to make pylint happy 2019-01-06 16:05:52 -05:00
Lex Berezhny d72a152195 run transaction interactions now return a result 2018-12-15 15:53:39 -05:00
Lex Berezhny 390226d2ef pylint 2018-12-15 15:53:39 -05:00
Lex Berezhny a16495d06f await self.server.stop() 2018-12-15 15:53:39 -05:00
Lex Berezhny 7092f40701 improved shutdown for torba.server and related test setup code 2018-12-15 15:53:39 -05:00
Victor Shyba 458189366f typo on session.py 2018-12-12 20:13:34 -03:00
Victor Shyba 9d772a45d5 wait the shutdown event instead of running forever 2018-12-12 20:13:08 -03:00
Victor Shyba 1c860e8578 set logging to info 2018-12-12 20:12:35 -03:00
Lex Berezhny 50fc027289 initial version of torba/server/cli.py 2018-12-12 12:54:38 -05:00
Victor Shyba e310601266 keep connection alive for idling peers 2018-12-11 16:07:48 -05:00
Lex Berezhny 78fd434686 asyncio.sleep() -> sleep() and asyncio.Event() to Event() in order to reduce overall diff after dropping curio.py 2018-12-11 15:59:15 -05:00
Lex Berezhny a6de3a9642 completely dropped curio.py 2018-12-11 15:59:15 -05:00
Lex Berezhny 5ed478ed11 dropped everything in curio.py except TaskGroup so far, TaskGroup is next 2018-12-11 15:59:15 -05:00
Lex Berezhny b93f9d4c94 removing dependence on curio abstraction 2018-12-11 15:59:15 -05:00
Victor Shyba f4ec20a2e2 tentatively enable test_reorg for travis 2018-12-10 13:48:23 -05:00
Victor Shyba 8bc07fad32 fast and furious 2018-12-07 08:47:30 -05:00
Lex Berezhny 612066165a improved integration test for tx synchronization 2018-12-06 13:38:56 -05:00
Lex Berezhny b273d58a80 0.2.5 2018-12-06 00:19:35 -05:00
Lex Berezhny b2544139e6 stress test 2018-12-05 23:45:35 -05:00
Lex Berezhny 367ffdea1f may be the sync fix 2018-12-05 23:42:16 -05:00
Lex Berezhny 4c854dd5a7 0.2.4 2018-12-05 11:50:11 -05:00
Lex Berezhny e96b2f702d 0.2.3 2018-12-05 11:49:22 -05:00
Lex Berezhny cff22a8951 pylint line too long 2018-12-05 11:48:19 -05:00
Lex Berezhny 1b7c5a1373 update_history serially handles unique addresses 2018-12-05 11:48:19 -05:00
Lex Berezhny 2bd60021aa pylint 2018-12-05 11:48:19 -05:00
Lex Berezhny 2b8c06bda7 wip sync fixing 2018-12-05 11:48:19 -05:00
Lex Berezhny c36b4626a9 added missing imports 2018-12-05 01:11:55 -05:00
Lex Berezhny b5f245d9b6 made pylint and mypy happy 2018-12-05 01:11:55 -05:00
Lex Berezhny 899a6f0d4a merged aiorpcx into torba.rpc 2018-12-05 01:11:55 -05:00
Lex Berezhny ba5fc2a627 allow 3.6 for now 2018-11-30 10:40:03 -05:00
Lex Berezhny 9525bd6ca0 do not run 3.6 tests anymore 2018-11-29 16:39:59 -05:00
Lex Berezhny 6c4f6dd59a do not install plyvel on windows 2018-11-29 16:28:38 -05:00
Jack Robison 997b573aa4 Merge pull request #42 from lbryio/catch-decrypt-errors
catch decryption errors and verify seed is valid upon decrypt
2018-11-26 14:53:33 -05:00
Jack Robison ca67c56222 catch decryption errors and verify seed is valid upon decrypt 2018-11-26 14:40:59 -05:00
Victor Shyba 56fc003a96 log errors from wallet servers 2018-11-26 00:32:20 -05:00
Lex Berezhny 8cb811ca1d force older astroid 2018-11-25 20:54:59 -05:00
Lex Berezhny 9e5246c619 drop old style classes in torba.server 2018-11-25 20:51:41 -05:00
Lex Berezhny b2bcf39872 0.2.2 2018-11-25 11:08:49 -05:00
Lex Berezhny 0fc864c2d5 update ignore path for bitcoin_headers 2018-11-25 11:08:13 -05:00
Lex Berezhny 4b39aad07e drop [test,server] extras from travis and tox.ini 2018-11-20 22:59:58 -05:00
Lex Berezhny 10c1f8e1cf drop requirements: mock, requests. merge [server] into the main requirement section 2018-11-20 22:57:59 -05:00
Lex Berezhny ed880be3e5 0.2.1 2018-11-19 19:27:07 -05:00
Lex Berezhny e5714dc1fc test sync integration test and other fixes 2018-11-19 18:14:57 -05:00
Lex Berezhny 1732167af1 use assertBalance in reconnect tests 2018-11-19 18:14:57 -05:00
Lex Berezhny ee28bbc19b pylint and mypy fixes 2018-11-19 18:14:57 -05:00
Lex Berezhny 345d4f8ab1 refactored tx sync 2018-11-19 18:14:57 -05:00
Jack Robison e745b6c16e fix wallet encryption
fixes https://github.com/lbryio/lbry/issues/1623
2018-11-19 17:15:09 -05:00
Victor Shyba ee57af8e99 restore skips for Travis 2018-11-07 16:46:47 -05:00
Victor Shyba 700f73e8e9 make tox run it all together 2018-11-07 16:46:47 -05:00
Victor Shyba fb6a7442a6 fix wallet server shutdown for tests 2018-11-07 16:46:47 -05:00
Lex Berezhny b8a3b62f0f moved import to make pylint happy, take 2 2018-11-07 15:10:27 -05:00
Lex Berezhny 78b84ba698 moved import to make pylint happy 2018-11-07 15:07:36 -05:00
Lex Berezhny fdf479241a added ledger.wait(tx) to support reliably waiting for TX to be in ledger 2018-11-07 14:42:17 -05:00
Lex Berezhny 8fcbf48bdf attempt UTXO check 5 times when UTXOs are less than the fee 2018-11-06 19:27:48 -05:00
Lex Berezhny 268e599423 handle creating transactions where input is consumed by fees and there is no output by creating an additional input which will require some change output 2018-11-06 19:16:27 -05:00
Lex Berezhny ae8fece9bb python3.6 tests pass again 2018-11-04 01:42:27 -05:00
Lex Berezhny c91c69540b orchstr8 command for orchestration 2018-11-04 01:09:16 -05:00
Lex Berezhny c497385bc7 setup.py fixes 2018-11-04 01:04:14 -05:00
Lex Berezhny 37468cad59 fix pylint 2018-11-04 01:01:03 -05:00
Lex Berezhny 5f70ae9923 reorganization 2018-11-04 01:55:50 -04:00
Lex Berezhny c1b96a36a7 skip running 3.6 tests 2018-11-03 20:52:26 -04:00
Lex Berezhny 5cdda0f3a8 fix AsyncTestCase imports 2018-11-03 20:52:26 -04:00
Lex Berezhny f0e1fc1a49 more pylint and mypy fixes 2018-11-03 20:52:26 -04:00
Lex Berezhny 77d8c68b13 pylint disable workbench check for now 2018-11-03 20:52:26 -04:00
Lex Berezhny 48d7bfd225 pylint and mypy fixes 2018-11-03 20:52:26 -04:00
Lex Berezhny f60435c878 moving and refactoring electrumx into torba 2018-11-03 20:52:26 -04:00
Victor Shyba 41332f22c3 bump aiorpcx + adapt basenetwork 2018-11-03 11:00:23 -04:00
Lex Berezhny a6ff18c872 pin aiorpcx to 0.8.2 until we can upgrade to latest 2018-10-26 16:20:07 -04:00
Lex Berezhny b90378d739 0.1.6 2018-10-25 17:49:17 -04:00
Victor Shyba a5a0089daa fix subscribe 2018-10-25 17:47:45 -04:00
Lex Berezhny 82c71e4255 v0.1.5 2018-10-21 11:53:13 -04:00
Victor Shyba dce5541c3a revert insert many due to lbrynet 2018-10-20 16:48:02 -04:00
Victor Shyba 8bbda75865 save_transaction_io back to a transaction 2018-10-20 16:48:02 -04:00
Nikita Titov e650562758 Use README as long description on PyPI 2018-10-20 10:17:02 -04:00
Lex Berezhny d3d60071ae v0.1.4 2018-10-19 11:01:35 -04:00
Victor Shyba 85503cf92b retry with a delay increasing from 0 up to 10 seconds 2018-10-19 10:59:27 -04:00
Lex Berezhny f67b872bd1 0.1.3 2018-10-18 18:21:39 -04:00
Victor Shyba c497e9884a pylint 2018-10-18 18:20:09 -04:00
Victor Shyba 3b021af1bd adds a simple sqlite asyncio wrapper 2018-10-18 18:20:09 -04:00
Lex Berezhny 6fe648b9a9 v0.1.2 2018-10-17 21:21:21 -04:00
Lex Berezhny 03594dab6a exception message correction 2018-10-17 21:20:21 -04:00
Victor Shyba b830f4c3ca fixes from code review 2018-10-17 19:43:26 -04:00
Victor Shyba 28b638f0aa prefetch and remove manual update calls 2018-10-17 19:43:26 -04:00
Victor Shyba 4d0ecd1fb4 fix history hash 2018-10-17 19:43:26 -04:00
Victor Shyba b62c5c57af re-enable test_reconnect on tox 2018-10-17 14:12:08 -04:00
Victor Shyba c7b647b1eb fix test_reconnect 2018-10-17 14:12:08 -04:00
Victor Shyba 63563c412c cancel pending requests and close connection on server switch 2018-10-17 14:12:08 -04:00
Victor Shyba 9fed3fce3e unlock networking limits 2018-10-17 14:01:54 -04:00
Lex Berezhny a8f8ff30a2 account list --show_seed fix 2018-10-17 10:57:24 -04:00
Lex Berezhny d270e84810 v0.1.1. 2018-10-17 10:39:21 -04:00
Lex Berezhny 0b8c67f2f0 move db.commit() 2018-10-16 21:56:50 -04:00
Lex Berezhny 2b73d3d109 pylint fixes 2018-10-16 21:53:23 -04:00
Lex Berezhny a7838b80c3 removed extraneous db.commit() 2018-10-16 21:49:49 -04:00
Lex Berezhny f2f2a12ea1 added commit() after insert/update operations 2018-10-16 21:45:59 -04:00
Lex Berezhny 382ebb55b9 log formatting fix 2018-10-16 11:56:53 -04:00
Lex Berezhny c6b32fa946 basedatabase.py test coverage 2018-10-15 17:15:05 -04:00
Lex Berezhny ae0e1bfaef support python 3.6 2018-10-15 17:15:05 -04:00
Lex Berezhny aa73cb9bbf elctrumx is needed by orchstr8 even in unittests 2018-10-15 17:15:05 -04:00
Lex Berezhny d60b98236b orchstr8 is needed by unittests now as well 2018-10-15 17:15:05 -04:00
Lex Berezhny 2cd1947ef5 fixing tests 2018-10-15 17:15:05 -04:00
Lex Berezhny e10f1df321 pylint and mypy fixes 2018-10-15 17:15:05 -04:00
Lex Berezhny 2c5fd4aade twisted -> asyncio 2018-10-15 17:15:05 -04:00
Victor Shyba 0ce4b9a7de tentatively refactor prefetch 2018-10-11 17:48:55 -04:00
Victor Shyba 44b1178cfe fix prefetch proofs for unconfirmed and separate prefetch logic 2018-10-11 17:48:55 -04:00
Victor Shyba 7bdcaa6737 sync: fetch proofs and txs in batches 2018-10-11 17:48:55 -04:00
Victor Shyba cab8981716 avoid reconstructing the history during loop 2018-10-11 17:48:55 -04:00
Victor Shyba 36ad834d43 add reconnection test 2018-10-11 17:45:54 -04:00
Victor Shyba b999abb9ea improve request cancelling and timeouts during switches 2018-10-11 17:45:54 -04:00
Victor Shyba 55f5c7491c fixes from review for readability 2018-10-11 17:41:26 -04:00
Victor Shyba 52b7bb0fce add test for checking db records when generating keys above the batch threshold 2018-10-11 17:41:26 -04:00
Victor Shyba b19fd8ed85 lower batch size so it works on restricted platforms
Windows and maybe some other platforms have a lower limit on sqlite variables.
2018-10-11 17:41:26 -04:00
Victor Shyba 6bdd9c9d78 optimize insert key loop 2018-10-11 17:41:26 -04:00
Victor Shyba 87c210952b enable WAL 2018-10-11 17:41:26 -04:00
Victor Shyba 4b9c3e8409 generate keys in batches of 2000s 2018-10-11 17:41:26 -04:00
Lex Berezhny c6f2d8b6fc fix pylint 2018-10-10 21:39:36 -04:00
Lex Berezhny b23481bd21 add height to TXRef 2018-10-10 21:29:29 -04:00
Lex Berezhny 64a626cbb5 v0.1.0 2018-10-09 23:59:57 -04:00
Lex Berezhny d4f964ef8e short circuit adding annotations when get_transaction does not return anything 2018-10-09 23:57:41 -04:00
Lex Berezhny 7f09405335 txi (txoid) index added 2018-10-09 23:39:00 -04:00
Lex Berezhny e37da81dc6 added database indexes 2018-10-09 22:52:43 -04:00
Lex Berezhny febadc5dd6 fix get_addresses where it was returned all addresses instead of for specific account 2018-10-09 20:36:58 -04:00
Lex Berezhny 4aea68ecd1 + account.get_*_count() methods 2018-10-09 19:29:20 -04:00
Lex Berezhny 6a97f96745 fixed database unit test 2018-10-09 16:49:37 -04:00
Lex Berezhny 3ed03c267a v0.0.9 2018-10-09 14:44:49 -04:00
Lex Berezhny c60e17b341 fix LIMIT OFFSET syntax error 2018-10-09 14:44:26 -04:00
Lex Berezhny 689c578081 get_or_create_usable_address returns a random choice when returning an existing unused address 2018-10-09 13:13:46 -04:00
Lex Berezhny 2723a55766 start() should not return until ledger has finished starting, which means self.join_network() needs to be yielded on the first connection 2018-10-08 23:27:32 -04:00
Lex Berezhny d02da2f3a5 fix test method name to make sure it gets run 2018-10-08 23:26:45 -04:00
Lex Berezhny 462fd8bad1 get_address test fixes 2018-10-08 23:26:11 -04:00
Lex Berezhny 75dd93436c more get_address fixes 2018-10-08 22:49:50 -04:00
Lex Berezhny eebfe2be08 get_address fix 2018-10-08 22:44:30 -04:00
Lex Berezhny 3a264c1eec drop ledger.get_transaction() in favor of just using ledger.db.get_transaction() directly 2018-10-08 18:50:45 -04:00
Victor Shyba 9c4dfa7f84 attach subscriptions to on_connected and stop service before discarding the object 2018-10-08 17:11:50 -04:00
Lex Berezhny d9c21ce37b v0.0.8 2018-10-08 14:53:57 -04:00
Lex Berezhny 1d84440b74 select_txos needs a join on tx table 2018-10-08 10:53:53 -04:00
Lex Berezhny 9aa22e1071 sql: pop offset/limit/order_by when doing simple count(*) query 2018-10-08 10:37:52 -04:00
Lex Berezhny 9cea8d2725 db.get_utxos_count -> db.get_utxo_count 2018-10-07 20:14:27 -04:00
Lex Berezhny 944238d542 removed unused method and minor pylint fix 2018-10-07 14:59:27 -04:00
Lex Berezhny 95cbdcb131 refactored sql queries, added full pagination support: limit, offset and count 2018-10-07 14:53:44 -04:00
Lex Berezhny 38b1fb5fba fix get_transaction unit test 2018-10-04 19:42:29 -04:00
Lex Berezhny fe59ed0105 remove unused import 2018-10-04 19:33:14 -04:00
Lex Berezhny 356ab9666f refactored get_transactions and get_txos 2018-10-04 19:27:39 -04:00
Lex Berezhny 0960762694 sql constraints support table.column dot delimited column names 2018-10-03 11:51:42 -04:00
Lex Berezhny 1f4a9cff26 ignore too-many-branches pylint issue 2018-10-03 09:41:52 -04:00
Lex Berezhny a6f97dfbde refactored queries 2018-10-03 09:41:52 -04:00
Lex Berezhny 04aa559037 simplified .gitignore 2018-10-02 09:53:23 -04:00
Lex Berezhny a55feee960 0.0.5 2018-10-01 22:03:48 -04:00
Lex Berezhny 29c0e3f15f fix for get_transaction 2018-09-25 23:28:06 -04:00
Victor Shyba b0bc06ae00 improve reorgs 2018-09-25 23:07:32 -04:00
Lex Berezhny cdbb34e976 pylint fix 2018-09-25 22:54:18 -04:00
Lex Berezhny 5c5b0ace53 mypy related fix 2018-09-25 22:46:22 -04:00
Lex Berezhny a879c822c2 check if txi.txo_ref.txo is not None before using it 2018-09-25 22:39:20 -04:00
Lex Berezhny 841d49554c fix unit tests 2018-09-25 18:21:42 -04:00
Lex Berezhny 0d22b8c23c typing and pylint fixes 2018-09-25 18:14:28 -04:00
Lex Berezhny c29b4c476d + tx.position, + tx.net_account_balance, + txo.is_my_account 2018-09-25 18:02:50 -04:00
Lex Berezhny 5977f42a7e remove unused import 2018-09-24 23:20:14 -04:00
Lex Berezhny 58d2c04b9f simplified and cleaned up typing for encryp / decrypt 2018-09-24 23:20:14 -04:00
Jack Robison c45c792657 add account encryption tests, update aes tests 2018-09-24 23:20:14 -04:00
Jack Robison c0e0b4b745 review feedback and pylint 2018-09-24 23:20:14 -04:00
Jack Robison aa47f10602 add encrypted_on_disk to BaseAccount
-encrypt the seed and private key upon to_dict() if encrypted_on_disk is true
2018-09-24 23:20:14 -04:00
Jack Robison 9268b6ca13 fix double-double sha 2018-09-24 23:20:14 -04:00
Jack Robison 4832526477 fix aes_encrypt not prepending the iv
-allow random iv to be overridden
2018-09-24 23:20:14 -04:00
Lex Berezhny 53e0fe9a69 pylint fix 2018-09-21 22:26:07 -04:00
Lex Berezhny 8ed0791b26 improved db.get_transactions() 2018-09-21 22:18:30 -04:00
Victor Shyba 8a87195f55 ecdsa -> coincurve 2018-09-21 20:05:59 -04:00
Lex Berezhny 0b75bb4052 + added get_transactions 2018-09-21 09:47:31 -04:00
Lex Berezhny 543f533a68 update bitcoin to 0.16.3 as 0.16.0 is no longer downloadable 2018-09-20 13:52:00 -04:00
Lex Berezhny 89f4ec12e9 pylint cleanup 2018-09-20 13:52:00 -04:00
Thomas Zarebczan d76758bd1d am I doing this right?
Copied from https://github.com/lbryio/lbry/commit/f1e36823025199461836733e2a1b2e22db35596e, looks like it could work, but didn't test locally.
2018-09-20 13:52:00 -04:00
Lex Berezhny 18629c93a2 defer.returnValue -> plain return per twisted support for Py3 plain return 2018-09-20 13:00:38 -04:00
Lex Berezhny b3f3fdeb1d explicit import of english words 2018-09-18 23:06:56 -04:00
Lex Berezhny 71d99046d1 dont perform pylint check in words directory 2018-09-17 22:59:24 -04:00
Lex Berezhny eedc9bf9ed store mnemonic words in python files instead of txt, this avoids packaging issues 2018-09-17 22:59:24 -04:00
Victor Shyba 94b920ab2a fix reconnect 2018-09-17 14:34:23 -04:00
Victor Shyba 873437e6a9 check for None on height to allow for genesis 2018-08-31 10:46:03 -04:00
Lex Berezhny 57abad2181 removed unnecessary address subscribe call in integration tests 2018-08-30 15:06:10 -04:00
Lex Berezhny 91d95bd42e have to explicitly subscribe accounts 2018-08-30 14:40:44 -04:00
Lex Berezhny b102afd68a - extraneous constants import 2018-08-30 12:46:51 -04:00
Lex Berezhny 9e55694512 + get_detailed_accounts 2018-08-30 11:50:11 -04:00
Lex Berezhny 00cf8131ec removed excessive white space around logo 2018-08-24 17:02:15 -04:00
Lex Berezhny 026bcd4d4d pylint fix 2018-08-23 02:56:25 -04:00
Lex Berezhny a27f89157e fixed bug where txis were duplicated in the database 2018-08-23 02:53:14 -04:00
Lex Berezhny 9a2e9d68e7 + PUSH_INTEGER 2018-08-20 22:04:22 -04:00
Lex Berezhny c961dd609a pylint fixes 2018-08-16 21:46:02 -04:00
Lex Berezhny dabedc17d0 improvements to reorg support 2018-08-16 21:41:22 -04:00
Lex Berezhny e7dcd9216b fixing tests 2018-08-16 01:21:45 -04:00
Lex Berezhny 95a22831aa unit test fixes 2018-08-16 01:11:03 -04:00
Lex Berezhny ac2fe3476a fixed unit tests in test_transaction.py 2018-08-16 01:03:18 -04:00
Lex Berezhny df251def26 pylint and mypy fixes 2018-08-16 00:56:46 -04:00
Lex Berezhny 133a86cd89 blockchain reorg handling and overall header refactor 2018-08-16 00:43:38 -04:00
Lex Berezhny 1a5654d50b Seed version 2018-08-15 17:26:09 -04:00
Lex Berezhny 5b28e77ab5 name conflict 2018-08-07 21:47:30 -04:00
Lex Berezhny f8c1790558 pylint and added type info 2018-08-07 21:36:44 -04:00
Lex Berezhny e2537e43b5 gave accounts reference to wallet so that they can save themselves after being modified 2018-08-07 21:31:29 -04:00
Lex Berezhny d4107600a5 from_dict() now also handles from_seed() 2018-08-06 02:52:52 -04:00
Lex Berezhny e996717b60 oops, basemanager shouldnt create default accounts, moved that into lbrynet manager 2018-08-06 00:26:46 -04:00
Lex Berezhny 75ddf209e2 create default account when missing and new new account.fund() to support transfer of funds between accounts 2018-08-06 00:15:19 -04:00
Lex Berezhny f5515e5e77 general purpose constraints builder 2018-08-03 21:26:53 -04:00
Lex Berezhny 1eb375ba80 add tx.fee property back 2018-08-03 12:29:02 -04:00
Lex Berezhny d3a171e638 + closest_match step between branch_and_bound and random_draw coin selections, + Transaction.create() single method instead of .pay()/.liquidate() also its more powerful 2018-08-03 10:41:40 -04:00
Lex Berezhny e154c83ffe pylint and mypy fixes 2018-07-29 14:34:56 -04:00
Lex Berezhny 69ad8e384a customizable address generator support 2018-07-29 13:13:40 -04:00
Lex Berezhny cfeb7b249b enable caching 2018-07-28 23:49:51 -04:00
Lex Berezhny 3a35951b46 travis yaml w/ anchors 2018-07-28 23:43:17 -04:00
Lex Berezhny 420ba43d5a travis yaml parse error fix 2018-07-28 23:02:12 -04:00
Lex Berezhny 2a42bda0a0 .travis.yml 2018-07-28 22:49:16 -04:00
Lex Berezhny 6be95dceab split unit tests and integration tests 2018-07-28 21:08:04 -04:00
Lex Berezhny a20c060931 install pylint from git 2018-07-28 20:57:50 -04:00
Lex Berezhny b0bd0b1fc0 + reserving outpoints should no longer have race conditions
+ converted all comment type annotations to python 3 style syntax annotations
+ pylint & mypy
2018-07-28 20:52:54 -04:00
Lex Berezhny 746aef474a hash.py dropping py2 backwards compat 2018-07-26 14:35:44 -04:00
Lex Berezhny 5a2a3727b2 + Account.get_max_gap() 2018-07-25 23:29:41 -04:00
Lex Berezhny 40651322d0 add account id to account balance listing 2018-07-25 12:05:25 -04:00
Lex Berezhny c2a9027c19 python 3.6 is also supported 2018-07-24 04:02:53 -04:00
Lex Berezhny 25046124c0 fix balance with confirmations and header event reporting 2018-07-22 22:52:21 -04:00
Lex Berezhny 2f83ebb3b5 tox fix 2018-07-21 16:12:25 -04:00
Lex Berezhny a03069bf89 travis-ci sudo:true 2018-07-21 16:02:38 -04:00
Lex Berezhny 93555cad3c ask travis to use xenial 2018-07-21 15:57:26 -04:00
Lex Berezhny 0a2d916931 py37 2018-07-21 15:06:20 -04:00
Lex Berezhny a4519d3add 0.0.4 2018-07-21 14:43:47 -04:00
Lex Berezhny 67dc297c97 use floats() instead of Decimal() when we need to serialized to JSON (while providing satoshis for accuracy) 2018-07-21 14:39:53 -04:00
Lex Berezhny 7c39e55089 callLater breaks determinism 2018-07-17 00:09:02 -04:00
Lex Berezhny 3f17dd336b 0.0.3 2018-07-17 00:01:22 -04:00
Lex Berezhny 10fd654edc improved account balance support 2018-07-16 23:58:29 -04:00
Victor Shyba 686eb7b1f0 fix a failure for channels where we catch the wrong error 2018-07-15 17:41:52 -03:00
Lex Berezhny 97532fb300 don't delete lock if it's not in locks dict and other fun things 2018-07-15 16:04:11 -04:00
Lex Berezhny e36e4c6354 pubkey should stay as blob 2018-07-15 00:40:46 -04:00
Lex Berezhny fcc7177a9a unit tests for bcd_data_stream 2018-07-14 22:38:18 -04:00
Lex Berezhny 0f786a8c43 removed unused methods 2018-07-14 22:16:39 -04:00
Lex Berezhny 184d3a5910 switched from blob to text for most columns, added a new TXRef/TXORef referencing scheme 2018-07-14 21:34:07 -04:00
Lex Berezhny 9370b1d2fa switching most columns from blob to text 2018-07-14 17:47:51 -04:00
Lex Berezhny f46550262f added SingleKey account support 2018-07-14 17:47:18 -04:00
Lex Berezhny adec715010 + ledger.get_transaction() 2018-07-12 12:14:04 -04:00
Lex Berezhny 1ce5da80d6 unit test fixes take 2 2018-07-11 22:49:54 -04:00
Lex Berezhny 0ffec82e35 unit test fixes 2018-07-11 22:47:44 -04:00
Lex Berezhny 32ba594e55 ledger API cleanup 2018-07-11 22:37:15 -04:00
Lex Berezhny feac754925 reduce verbosity for integration test 2018-07-10 21:23:13 -04:00
Lex Berezhny 565b095a8d have to keep using old electrumx until we upgrade everywhere 2018-07-10 21:17:07 -04:00
Lex Berezhny 5db792e99c .travis.yml: electrumx master now contains the extensibility feature 2018-07-10 21:09:49 -04:00
Lex Berezhny c9c3585e1e test for Transaction.liquidate 2018-07-09 22:02:18 -04:00
Lex Berezhny 4a0d461446 use extensible branch of electrumx 2018-07-09 20:57:09 -04:00
Lex Berezhny 630a805c49 improved test coverage for bip32 2018-07-09 20:22:18 -04:00
Lex Berezhny 5eb276655d + Transaction.liquidate, balance/get_utxos support constraints 2018-07-09 20:22:04 -04:00
Lex Berezhny 2006ad68ee get_balance supports custom constraints 2018-07-09 09:55:59 -04:00
Lex Berezhny ae56b81a08 test manager.get_balance() 2018-07-04 21:30:38 -04:00
Lex Berezhny e2296d4491 improving code coverage 2018-07-04 21:17:45 -04:00
Lex Berezhny dd40d2e6ce explicit env for tox 2018-07-04 20:51:18 -04:00
Lex Berezhny c3cb297205 unit test fixes and reducing support for py2 2018-07-04 20:43:25 -04:00
Lex Berezhny 6bb81e4d6f wip4victor 2018-07-01 17:20:17 -04:00
Lex Berezhny c58a1c9ead BaseLedger.stop() should also close the db 2018-06-26 20:41:03 -04:00
Lex Berezhny da265198f1 cleanup integration test 2018-06-26 20:22:35 -04:00
Lex Berezhny 1dfc18683d fixing unit tests 2018-06-26 18:31:42 -04:00
Lex Berezhny 9a467f8840 * on_transaction now produces TransactionEvents with useful info and lots of other changes 2018-06-26 17:22:05 -04:00
Lex Berezhny aba3ed7ca0 SPV support 2018-06-25 09:54:35 -04:00
Lex Berezhny 22b897db2e unit test fix 2018-06-17 23:29:23 -04:00
Lex Berezhny 833ef98ff5 refactoring and added basic test for reading/writing wallet file 2018-06-17 23:22:15 -04:00
Lex Berezhny 43cd9c4100 removed unused msqr module 2018-06-14 16:27:17 -04:00
Lex Berezhny 4dee59a49d better customization during testing 2018-06-14 15:17:59 -04:00
Lex Berezhny 52846a3b24 switch to master branch of orchstr8 2018-06-14 00:03:52 -04:00
Lex Berezhny ac0c88768d moved orchstr8 download into tox.ini instead of .travis.yml 2018-06-13 23:00:53 -05:00
Lex Berezhny 9977a398af orchstr8 download needs electrumx installed 2018-06-13 23:00:53 -05:00
Lex Berezhny 643998332e orchstr8 download needs torba installed 2018-06-13 23:00:53 -05:00
Lex Berezhny 13444ae6c3 try downloading blockchain binary before tests run to prevent timeout during testing 2018-06-13 23:00:53 -05:00
Lex Berezhny 05ec13394c only run py36 and bitcoinsegwit integration tests until we work out the kinks 2018-06-13 23:00:53 -05:00
Lex Berezhny b7cbe16159 spelling fix and cleaned up tox.ini 2018-06-13 23:00:53 -05:00
Lex Berezhny f0ecf58752 use wip branch of orchstr8 2018-06-13 23:00:53 -05:00
Lex Berezhny e9fbc875e0 passing tests 2018-06-13 23:00:53 -05:00
Lex Berezhny 99ae39012f tests/__init__.py 2018-06-13 23:00:53 -05:00
Lex Berezhny fbd1b2ec30 monday progress 2018-06-13 23:00:53 -05:00
Lex Berezhny eb6781481a progress since thursday 2018-06-13 23:00:53 -05:00
Lex Berezhny 78e4135159 wip 2018-06-13 23:00:53 -05:00
Lex Berezhny 54350bcbc4 BaseOutputAmountEstimator 2018-06-03 20:13:30 -04:00
Lex Berezhny 09a32174c8 handle tuples in network RPC request data 2018-06-03 20:07:40 -04:00
Lex Berezhny 740af0465e send_amount_to_address 2018-06-03 20:06:55 -04:00
Lex Berezhny 08f3ecbf25 initial readme 2018-06-01 01:52:51 -04:00
Lex Berezhny 02768a8181 shorter icon 2018-06-01 01:34:30 -04:00
Lex Berezhny c86113b288 torba icon 2018-06-01 01:32:12 -04:00
Lex Berezhny 246cc234c4 test fixes to get it running on travis 2018-05-25 13:19:54 -04:00
Lex Berezhny 4e4c4b5f96 fixing travis 2018-05-25 11:11:28 -04:00
Lex Berezhny ca6d5937ba stream.where and stream.async_where 2018-05-25 09:54:01 -04:00
Lex Berezhny ece2db08da initial import 2018-05-25 02:03:25 -04:00
521 changed files with 70401 additions and 40984 deletions

No files matched your search

-7
View File
@@ -1,7 +0,0 @@
.git
.tox
__pycache__
dist
lbrynet.egg-info
docs
tests
-1
View File
@@ -1 +0,0 @@
/CHANGELOG.md merge=union
-74
View File
@@ -1,74 +0,0 @@
<!--
Thanks for reporting an issue to LBRY and helping us improve!
To make it possible for us to help you, please fill out below information carefully.
Before reporting any issues, please make sure that you're using the latest version.
- App: https://github.com/lbryio/lbry-desktop/releases
- Daemon: https://github.com/lbryio/lbry/releases
We are also available on Discord at https://chat.lbry.io
-->
## The Issue
In order to <achieve some value>,
as a <type of user>,
I want <some functionality>.
### Steps to reproduce
1.
2.
3.
### Expected behaviour
Tell us what should happen
### Actual behaviour
Tell us what happens instead
## System Configuration
<!-- For the app, this info is in the About section at the bottom of the Help page.
You can include a screenshot instead of typing it out -->
<!-- For the daemon, run:
curl 'http://localhost:5279' --data '{"method":"version"}'
and include the full output -->
- LBRY Daemon version:
- LBRY App version:
- LBRY Installation ID:
- Operating system:
## Anything Else
<!-- Include anything else that does not fit into the above sections -->
## Screenshots
<!-- If a screenshot would help explain the bug, please include one or two here -->
## Internal Use
### Acceptance Criteria
1.
2.
3.
### Definition of Done
- [ ] Tested against acceptance criteria
- [ ] Tested against the assumptions of user story
- [ ] The project builds without errors
- [ ] Unit tests are written and passing
- [ ] Tests on devices/browsers listed in the issue have passed
- [ ] QA performed & issues resolved
- [ ] Refactoring completed
- [ ] Any configuration or build changes documented
- [ ] Documentation updated
- [ ] Peer Code Review performed
-38
View File
@@ -1,38 +0,0 @@
## PR Checklist
Please check all that apply to this PR using "x":
- [ ] I have checked that this PR is not a duplicate of an existing PR (open, closed or merged)
- [ ] I have checked that this PR does not introduce a breaking change
- [ ] This PR introduces breaking changes and I have provided a detailed explanation below
## PR Type
What kind of change does this PR introduce?
Why is this change necessary?
<!-- Please check all that apply to this PR using "x". -->
- [ ] Bugfix
- [ ] Feature
- [ ] Breaking changes (bugfix or feature that introduces breaking changes)
- [ ] Code style update (formatting)
- [ ] Refactoring (no functional changes)
- [ ] Documentation changes
- [ ] Other - Please describe:
## Fixes
Issue Number: N/A
## What is the current behavior?
## What is the new behavior?
## Other information
<!-- If this PR contains a breaking change, please describe the impact and solution strategy for existing applications below. -->
+197
View File
@@ -0,0 +1,197 @@
name: ci
on: push
jobs:
lint:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v1
with:
python-version: '3.7'
- name: extract pip cache
uses: actions/cache@v2
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}
restore-keys: ${{ runner.os }}-pip-
- run: |
pip install --user --upgrade pip wheel
pip install -e .[lint]
- run: make lint
tests-unit:
name: "tests / unit"
strategy:
matrix:
os:
- ubuntu-latest
- macos-latest
- windows-latest
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v1
with:
python-version: '3.7'
- name: set pip cache dir
id: pip-cache
run: echo "::set-output name=dir::$(pip cache dir)"
- name: extract pip cache
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}
restore-keys: ${{ runner.os }}-pip-
- run: |
pip install --user --upgrade pip wheel
pip install -e .[test]
- env:
HOME: /tmp
run: coverage run -m unittest discover -v tests.unit
- env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COVERALLS_PARALLEL: true
name: Submit to coveralls
run: |
pip install https://github.com/bboe/coveralls-python/archive/github_actions.zip
coveralls
tests-integration:
name: "tests / integration"
runs-on: ubuntu-20.04
strategy:
matrix:
test:
# - datanetwork
- blockchain
# - other
db:
- sqlite
- postgres
services:
postgres:
image: postgres:12
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v1
with:
python-version: '3.7'
- if: matrix.test == 'other'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends ffmpeg
- name: extract pip cache
uses: actions/cache@v2
with:
path: ./.tox
key: tox-integration-${{ matrix.test }}-${{ matrix.db }}-${{ hashFiles('setup.py') }}
restore-keys: txo-integration-${{ matrix.test }}-${{ matrix.db }}-
- run: pip install tox
- env:
TEST_DB: ${{ matrix.db }}
run: tox -e ${{ matrix.test }}
- env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COVERALLS_PARALLEL: true
name: Submit to coveralls
run: |
pip install https://github.com/bboe/coveralls-python/archive/github_actions.zip
coverage combine tests
coveralls
coveralls-finished:
needs: ["tests-unit", "tests-integration"]
runs-on: ubuntu-latest
steps:
- name: Coveralls Finished
uses: coverallsapp/github-action@57daa114
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
parallel-finished: true
build:
needs: ["lint", "tests-unit", "tests-integration"]
name: "build"
strategy:
matrix:
os:
- ubuntu-16.04
- macos-latest
- windows-latest
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v1
with:
python-version: '3.7'
- name: set pip cache dir
id: pip-cache
run: echo "::set-output name=dir::$(pip cache dir)"
- name: extract pip cache
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}
restore-keys: ${{ runner.os }}-pip-
- name: Setup
run: |
pip install --user --upgrade pip wheel
pip install sqlalchemy@git+https://github.com/eukreign/pyinstaller.git@sqlalchemy
- if: startsWith(runner.os, 'linux')
run: |
sudo apt-get install libzmq3-dev
pip install -e .[postgres]
- if: startsWith(runner.os, 'mac')
run: |
brew install zeromq
pip install -e .
- if: startsWith(runner.os, 'linux') || startsWith(runner.os, 'mac')
name: Build & Run (Unix)
run: |
pyinstaller --onefile --name lbrynet lbry/cli.py
chmod +x dist/lbrynet
dist/lbrynet --version
- if: startsWith(runner.os, 'windows')
name: Build & Run (Windows)
run: |
pip install pywin32
pip install -e .
pyinstaller --additional-hooks-dir=scripts/. --icon=icons/lbry256.ico --onefile --name lbrynet lbry/cli.py
dist/lbrynet.exe --version
- uses: actions/upload-artifact@v2
with:
name: lbrynet-${{ matrix.os }}
path: dist/
docker:
needs: ["build"]
name: "build (docker)"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: fetch lbrynet binary
uses: actions/download-artifact@v2
with:
name: lbrynet-ubuntu-16.04
- run: |
chmod +x lbrynet
mv lbrynet docker
- name: build and push docker image
uses: docker/build-push-action@v1
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
repository: lbry/lbrynet
path: docker
tag_with_ref: true
tag_with_sha: true
add_git_labels: true
+12 -3
View File
@@ -1,9 +1,18 @@
/.idea
/.DS_Store
/build
/dist
/.tox
/.idea
/.coverage
/.coverage*
/lbry-venv
/venv
lbrynet.egg-info
lbry.egg-info
__pycache__
_trial_temp/
/tests/integration/commands/files
/tests/.coverage.*
/lbry/blockchain/bin
/lbry/wallet/bin
+210
View File
@@ -0,0 +1,210 @@
default:
image: python:3.7
#cache:
# directories:
# - $HOME/venv
# - $HOME/.cache/pip
# - $HOME/Library/Caches/pip
# - $HOME/Library/Caches/Homebrew
# - $TRAVIS_BUILD_DIR/.tox
stages:
- test
- build
- assets
- release
.tagged:
rules:
- if: '$CI_COMMIT_TAG =~ /^v[0-9\.]+$/'
when: on_success
test:lint:
stage: test
script:
- make install tools
- make lint
test:unit:
stage: test
script:
- make install tools
- HOME=/tmp coverage run -p --source=lbry -m unittest discover -vv tests.unit
test:datanetwork-integration:
stage: test
script:
- pip install tox-travis
- tox -e datanetwork
test:blockchain-integration:
stage: test
script:
- pip install tox-travis
- tox -e blockchain
test:other-integration:
stage: test
script:
- apt-get update
- apt-get install -y --no-install-recommends ffmpeg
- pip install tox-travis
- tox -e other
test:json-api:
stage: test
script:
- make install tools
- HOME=/tmp coverage run -p --source=lbry scripts/generate_json_api.py
.build:
stage: build
artifacts:
expire_in: 1 day
paths:
- lbrynet-${OS}.zip
script:
- pip install --upgrade 'setuptools<45.0.0'
- pip install pyinstaller
- pip install -e .
- python3.7 docker/set_build.py # must come after lbry is installed because it imports lbry
- pyinstaller --onefile --name lbrynet lbry/extras/cli.py
- chmod +x dist/lbrynet
- zip --junk-paths ${CI_PROJECT_DIR}/lbrynet-${OS}.zip dist/lbrynet # gitlab expects artifacts to be in $CI_PROJECT_DIR
- openssl dgst -sha256 ${CI_PROJECT_DIR}/lbrynet-${OS}.zip | egrep -o [0-9a-f]+$ # get sha256 of asset. works on mac and ubuntu
- dist/lbrynet --version
build:linux:
extends: .build
image: ubuntu:16.04
variables:
OS: linux
before_script:
- apt-get update
- apt-get install -y --no-install-recommends software-properties-common zip curl build-essential
- add-apt-repository -y ppa:deadsnakes/ppa
- apt-get update
- apt-get install -y --no-install-recommends python3.7-dev
- python3.7 <(curl -q https://bootstrap.pypa.io/get-pip.py) # make sure we get pip with python3.7
build:mac:
extends: .build
tags: [macos] # makes gitlab use the mac runner
variables:
OS: mac
GIT_DEPTH: 5
VENV: /tmp/gitlab-lbry-sdk-venv
before_script:
# - brew upgrade python || true
- python3 --version | grep -q '^Python 3\.7\.' # dont upgrade python on every run. just make sure we're on the right Python
# - pip3 install --user --upgrade pip virtualenv
- pip3 --version | grep -q '\(python 3\.7\)'
- virtualenv --python=python3.7 "${VENV}"
- source "${VENV}/bin/activate"
after_script:
- rm -rf "${VENV}"
build:windows:
extends: .build
tags: [windows] # makes gitlab use the windows runner
variables:
OS: windows
GIT_DEPTH: 5
before_script:
- ./docker/install_choco.ps1
- choco install -y --x86 python3 7zip checksum
- python --version # | findstr /B "Python 3\.7\." # dont upgrade python on every run. just make sure we're on the right Python
- pip --version # | findstr /E '\(python 3\.7\)'
- pip install virtualenv pywin32
- virtualenv venv
- venv/Scripts/activate.ps1
- pip install pip==19.3.1; $true # $true ignores errors. need this to get the correct coincurve wheel. see commit notes for details.
after_script:
- rmdir -Recurse venv
script:
- pip install --upgrade 'setuptools<45.0.0'
- pip install pyinstaller==3.5
- pip install -e .
- python docker/set_build.py # must come after lbry is installed because it imports lbry
- pyinstaller --additional-hooks-dir=scripts/. --icon=icons/lbry256.ico -F -n lbrynet lbry/extras/cli.py
- 7z a -tzip $env:CI_PROJECT_DIR/lbrynet-${OS}.zip ./dist/lbrynet.exe
- checksum --type=sha256 --file=$env:CI_PROJECT_DIR/lbrynet-${OS}.zip
- dist/lbrynet.exe --version
# s3 = upload asset to s3 (build.lbry.io)
.s3:
stage: assets
variables:
GIT_STRATEGY: none
script:
- "[ -f lbrynet-${OS}.zip ]" # check that asset exists before trying to upload
- pip install awscli
- S3_PATH="daemon/gitlab-build-${CI_PIPELINE_ID}_commit-${CI_COMMIT_SHA:0:7}$( if [ ! -z ${CI_COMMIT_TAG} ]; then echo _tag-${CI_COMMIT_TAG}; else echo _branch-${CI_COMMIT_REF_NAME}; fi )"
- AWS_ACCESS_KEY_ID=${ARTIFACTS_KEY} AWS_SECRET_ACCESS_KEY=${ARTIFACTS_SECRET} AWS_REGION=${ARTIFACTS_REGION}
aws s3 cp lbrynet-${OS}.zip s3://${ARTIFACTS_BUCKET}/${S3_PATH}/lbrynet-${OS}.zip
s3:linux:
extends: .s3
variables: {OS: linux}
needs: ["build:linux"]
s3:mac:
extends: .s3
variables: {OS: mac}
needs: ["build:mac"]
s3:windows:
extends: .s3
variables: {OS: windows}
needs: ["build:windows"]
# github = upload assets to github when there's a tagged release
.github:
extends: .tagged
stage: assets
variables:
GIT_STRATEGY: none
script:
- "[ -f lbrynet-${OS}.zip ]" # check that asset exists before trying to upload. githubrelease won't error if its missing
- pip install githubrelease
- githubrelease --no-progress --github-token ${GITHUB_CI_USER_ACCESS_TOKEN} asset lbryio/lbry-sdk upload ${CI_COMMIT_TAG} lbrynet-${OS}.zip
github:linux:
extends: .github
variables: {OS: linux}
needs: ["build:linux"]
github:mac:
extends: .github
variables: {OS: mac}
needs: ["build:mac"]
github:windows:
extends: .github
variables: {OS: windows}
needs: ["build:windows"]
publish:
extends: .tagged
stage: release
variables:
GIT_STRATEGY: none
script:
- pip install githubrelease
- githubrelease --no-progress --github-token ${GITHUB_CI_USER_ACCESS_TOKEN} release lbryio/lbry-sdk publish ${CI_COMMIT_TAG}
- >
curl -X POST -H 'Content-type: application/json' --data '{"text":"<!channel> There is a new SDK release: https://github.com/lbryio/lbry-sdk/releases/tag/'"${CI_COMMIT_TAG}"'\n'"$(curl -s "https://api.github.com/repos/lbryio/lbry-sdk/releases/tags/${CI_COMMIT_TAG}" | egrep '\w*\"body\":' | cut -d':' -f 2- | tail -c +3 | head -c -2)"'", "channel":"tech"}' "$(echo ${SLACK_WEBHOOK_URL_BASE64} | base64 -d)"
-447
View File
@@ -1,447 +0,0 @@
[MASTER]
# Specify a configuration file.
#rcfile=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS,schema
# Add files or directories matching the regex patterns to the
# blacklist. The regex matches against base names, not paths.
# `\.#.*` - add emacs tmp files to the blacklist
ignore-patterns=\.#.*
# Pickle collected data for later comparisons.
persistent=yes
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
# Use multiple processes to speed up Pylint.
jobs=1
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
miniupnpc,
unqlite
# Allow optimization of some AST trees. This will activate a peephole AST
# optimizer, which will apply various small optimizations. For instance, it can
# be used to obtain the result of joining multiple strings with the addition
# operator. Joining a lot of strings can lead to a maximum recursion error in
# Pylint and this flag can prevent that. It has one side effect, the resulting
# AST will be different than the one from reality.
optimize-ast=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
#enable=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then re-enable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=
anomalous-backslash-in-string,
arguments-differ,
attribute-defined-outside-init,
bad-continuation,
bare-except,
broad-except,
cell-var-from-loop,
consider-iterating-dictionary,
dangerous-default-value,
duplicate-code,
fixme,
global-statement,
inherit-non-class,
invalid-name,
len-as-condition,
locally-disabled,
logging-not-lazy,
missing-docstring,
no-else-return,
no-init,
no-member,
no-self-use,
protected-access,
redefined-builtin,
redefined-outer-name,
redefined-variable-type,
relative-import,
signature-differs,
super-init-not-called,
too-few-public-methods,
too-many-arguments,
too-many-branches,
too-many-instance-attributes,
too-many-lines,
too-many-locals,
too-many-nested-blocks,
too-many-public-methods,
too-many-return-statements,
too-many-statements,
trailing-newlines,
undefined-loop-variable,
ungrouped-imports,
unnecessary-lambda,
unused-argument,
unused-variable,
wildcard-import,
wrong-import-order,
wrong-import-position,
deprecated-lambda,
simplifiable-if-statement,
unidiomatic-typecheck,
global-at-module-level,
inconsistent-return-statements,
keyword-arg-before-vararg,
assignment-from-no-return,
useless-return,
assignment-from-none,
stop-iteration-return
[REPORTS]
# Set the output format. Available formats are text, parseable, colorized, msvs
# (visual studio) and html. You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
output-format=text
# Put messages in a separate file for each module / package specified on the
# command line instead of printing them on stdout. Reports (if any) will be
# written in a file name "pylint_global.[txt|html]".
files-output=no
# Tells whether to display a full report or only the messages
reports=no
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
[VARIABLES]
# Tells whether we should check for unused import in __init__ files.
init-import=no
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_$|dummy
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,_cb
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[BASIC]
# List of builtins function names that should not be used, separated by a comma
bad-functions=map,filter,input
# Good variable names which should always be accepted, separated by a comma
# allow `d` as its used frequently for deferred callback chains
good-names=i,j,k,ex,Run,_,d
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,bar,baz,toto,tutu,tata
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Regular expression matching correct function names
function-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for function names
function-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct variable names
variable-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for variable names
variable-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct constant names
const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Naming hint for constant names
const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Regular expression matching correct attribute names
attr-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for attribute names
attr-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct argument names
argument-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for argument names
argument-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct class attribute names
class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Naming hint for class attribute names
class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Regular expression matching correct inline iteration names
inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
# Naming hint for inline iteration names
inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$
# Regular expression matching correct class names
class-rgx=[A-Z_][a-zA-Z0-9]+$
# Naming hint for class names
class-name-hint=[A-Z_][a-zA-Z0-9]+$
# Regular expression matching correct module names
module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Naming hint for module names
module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Regular expression matching correct method names
method-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for method names
method-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
[ELIF]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
[SPELLING]
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[FORMAT]
# Maximum number of characters on a single line.
max-line-length=120
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
# List of optional constructs for which whitespace checking is disabled. `dict-
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
# `empty-line` allows space-only lines.
no-space-check=trailing-comma,dict-separator
# Maximum number of lines in a module
max-module-lines=1000
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,XXX,TODO
[SIMILARITIES]
# Minimum lines number of a similarity.
min-similarity-lines=4
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=no
[TYPECHECK]
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=twisted.internet.reactor,leveldb,distutils
# Ignoring distutils because: https://github.com/PyCQA/pylint/issues/73
# List of classes names for which member attributes should not be checked
# (useful for classes with attributes dynamically set). This supports can work
# with qualified names.
ignored-classes=twisted.internet.reactor,RequestMessage
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=lbrynet.lbrynet_daemon.LBRYDaemon.Parameters
[IMPORTS]
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=regsub,TERMIOS,Bastion,rexec
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
[DESIGN]
# Maximum number of arguments for function / method
max-args=10
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of statements in function / method body
max-statements=50
# Maximum number of parents for a class (see R0901).
max-parents=8
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,__new__,setUp
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,_fields,_replace,_source,_make
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=Exception
-130
View File
@@ -1,130 +0,0 @@
sudo: required
dist: xenial
language: python
python: "3.7"
jobs:
include:
- stage: code quality
name: "pylint lbrynet"
install:
- pip install astroid==2.0.4 aiohttp==3.4.4
# newer astroid and aiohttp fails in pylint so we pre-install older version
- pip install pylint
- pip install git+https://github.com/lbryio/torba.git#egg=torba
- pip install -e .
script: pylint lbrynet
- &tests
stage: test
name: "Unit Tests"
install:
- pip install coverage
- pip install git+https://github.com/lbryio/torba.git#egg=torba
- pip install -e .[test]
script:
- HOME=/tmp coverage run -p --source=lbrynet -m twisted.trial --reactor=asyncio tests.unit.core tests.unit.cryptstream tests.unit.database tests.unit.dht tests.unit.lbryfilemanager tests.unit.lbrynet_daemon tests.unit.schema tests.unit.wallet tests.unit.components tests.unit.test_conf
- HOME=/tmp coverage run -p --source=lbrynet -m twisted.trial --reactor=asyncio tests.unit.test_cli
#- HOME=/tmp coverage run -p --source=lbrynet -m twisted.trial --reactor=asyncio tests.unit.analytics
after_success:
- coverage combine
- bash <(curl -s https://codecov.io/bash)
- <<: *tests
name: "DHT Tests"
script: HOME=/tmp coverage run --source=lbrynet -m twisted.trial --reactor=asyncio tests.functional
- name: "Integration Tests"
install:
- pip install tox-travis coverage
- pushd .. && git clone https://github.com/lbryio/torba.git && popd
script: tox
after_success:
- coverage combine tests/
- bash <(curl -s https://codecov.io/bash)
- stage: build
name: "Windows"
language: generic
services:
- docker
install:
- docker pull lbry/pyinstaller34_32bits:py371
script:
- python scripts/set_build.py
- docker run -v "$(pwd):/src/lbry" lbry/pyinstaller34_32bits:py371 lbry/scripts/wine_build.sh
- sudo zip -j dist/lbrynet-windows.zip dist/lbrynet.exe
deploy:
provider: releases
api_key: $GITHUB_OAUTH_TOKEN
file: dist/lbrynet-windows.zip
skip_cleanup: true
overwrite: true
draft: true
on:
tags: true
addons:
artifacts:
working_dir: dist
paths:
- lbrynet-windows.zip
target_paths:
- /daemon/build-${TRAVIS_BUILD_NUMBER}_commit-${TRAVIS_COMMIT:0:7}_branch-${TRAVIS_BRANCH}$([ ! -z ${TRAVIS_TAG} ] && echo _tag-${TRAVIS_TAG})
- &build
name: "Linux"
env: OS=linux
install:
- pip3 install pyinstaller
- pip3 install git+https://github.com/lbryio/torba.git
- python scripts/set_build.py
- pip3 install -e .
script:
- pyinstaller -F -n lbrynet lbrynet/extras/cli.py
- chmod +x dist/lbrynet
- zip -j dist/lbrynet-${OS}.zip dist/lbrynet
- ./dist/lbrynet --version
deploy:
provider: releases
api_key: $GITHUB_OAUTH_TOKEN
file: dist/lbrynet-${OS}.zip
skip_cleanup: true
overwrite: true
draft: true
on:
tags: true
addons:
artifacts:
working_dir: dist
paths:
- lbrynet-${OS}.zip
# artifact uploader thinks lbrynet is a directory, https://github.com/travis-ci/artifacts/issues/78
target_paths:
- /daemon/build-${TRAVIS_BUILD_NUMBER}_commit-${TRAVIS_COMMIT:0:7}_branch-${TRAVIS_BRANCH}$([ ! -z ${TRAVIS_TAG} ] && echo _tag-${TRAVIS_TAG})
- <<: *build
name: "Mac"
os: osx
osx_image: xcode8.3
language: generic
env: OS=mac
cache: false
before_install:
- brew upgrade python || true
- brew upgrade python || true
install:
- python3 --version
- pip3 --version
- pip3 install pyinstaller
- git clone https://github.com/lbryio/torba.git --depth 1
- sed -i -e "s/'plyvel',//" torba/setup.py
- cd torba && pip3 install -e . && cd ..
- python3 scripts/set_build.py
- pip3 install -e .
cache:
directories:
- $HOME/.cache/pip
- $HOME/Library/Caches/pip
- $TRAVIS_BUILD_DIR/.tox
+1 -1018
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,3 +1,3 @@
## Contributing to LBRY
https://lbry.io/faq/contributing
https://lbry.tech/contribute
-6
View File
@@ -1,6 +0,0 @@
# Add a CHANGELOG entry for app changes
has_app_changes = !git.modified_files.grep(/lbrynet/).empty?
if !git.modified_files.include?("CHANGELOG.md") && has_app_changes
fail("Please include a CHANGELOG entry.")
message "See http://keepachangelog.com/en/0.3.0/ for details on good changelog guidelines"
end
+54 -23
View File
@@ -1,6 +1,6 @@
# Installing LBRY
If only the JSON-RPC API server is needed, the recommended way to install LBRY is to use a pre-built binary. We provide binaries for all major operating systems. See the [README](README.md).
If only the JSON-RPC API server is needed, the recommended way to install LBRY is to use a pre-built binary. We provide binaries for all major operating systems. See the [README](README.md)!
These instructions are for installing LBRY from source, which is recommended if you are interested in doing development work or LBRY is not available on your operating system (godspeed, TempleOS users).
@@ -9,35 +9,34 @@ Here's a video walkthrough of this setup, which is itself hosted by the LBRY net
## Prerequisites
Running `lbrynet` from source requires Python 3.6 or higher (3.7 is preferred). Get the installer for your OS [here](https://www.python.org/downloads/release/python-370/)
Running `lbrynet` from source requires Python 3.7 or higher. Get the installer for your OS [here](https://www.python.org/downloads/release/python-370/).
After installing python 3, you'll need to install some additional libraries depending on your operating system.
### Virtualenv
Once python 3 is installed run `python3 -m pip install virtualenv` to install virtualenv.
### Windows
Windows users will need to install `Visual C++ Build Tools`, which can be installed by [Microsoft Build Tools](Microsoft Build Tools 2015)
### macOS
macOS users will need to install [xcode command line tools](https://developer.xamarin.com/guides/testcloud/calabash/configuring/osx/install-xcode-command-line-tools/) and [homebrew](http://brew.sh/).
These environment variables also need to be set:
1. PYTHONUNBUFFERED=1
2. EVENT_NOKQUEUE=1
Remaining dependencies can then be installed by running:
```
brew install python3 protobuf
brew install python protobuf
```
Assistance installing Python3: https://docs.python-guide.org/starting/install3/osx/.
### Linux
On Ubuntu (we recommend 18.04), install the following:
On Ubuntu (16.04 minimum, we recommend 18.04), install the following:
```
sudo apt-get install build-essential python3.7 python3.7-dev git python-virtualenv libssl-dev python-protobuf
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install build-essential python3.7 python3.7-dev git python3.7-venv libssl-dev python-protobuf
```
On Raspbian, you will also need to install `python-pyparsing`.
@@ -46,26 +45,58 @@ If you're running another Linux distro, install the equivalent of the above pack
## Installation
To install:
### Linux/Mac
To install on Linux/Mac:
```
git clone https://github.com/lbryio/lbry.git
cd lbry
Clone the repository:
$ git clone https://github.com/lbryio/lbry-sdk.git
$ cd lbry-sdk
virtualenv lbry-venv --python=python3.7
source lbry-venv/bin/activate
Create a Python virtual environment for lbry-sdk:
$ python3.7 -m venv lbry-venv
Activating lbry-sdk virtual environment:
$ source lbry-venv/bin/activate
Make sure you're on Python 3.7+ (as the default Python in virtual environment):
$ python --version
python --version # Python 2 is not supported. Make sure you're on Python 3.7
Install packages:
$ make install
pip install -e .
If you are on Linux and using PyCharm, generates initial configs:
$ make idea
```
To verify your installation, `which lbrynet` should return a path inside of the `lbry-venv` folder created by the `virtualenv` command.
To verify your installation, `which lbrynet` should return a path inside of the `lbry-venv` folder created by the `python3.7 -m venv lbry-venv` command.
### Windows
To install on Windows:
```
Clone the repository:
> git clone https://github.com/lbryio/lbry-sdk.git
> cd lbry-sdk
Create a Python virtual environment for lbry-sdk:
> python -m venv lbry-venv
Activating lbry-sdk virtual environment:
> lbry-venv\Scripts\activate
Install packages:
> pip install -e .
```
## Run the tests
To run the unit tests from the repo directory:
```
trial --reactor=asyncio tests.unit
python -m unittest discover tests.unit
```
## Usage
+1 -1
View File
@@ -1,6 +1,6 @@
The MIT License (MIT)
Copyright (c) 2015-2018 LBRY Inc
Copyright (c) 2015-2020 LBRY Inc
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish,
+4
View File
@@ -0,0 +1,4 @@
include README.md
include CHANGELOG.md
include LICENSE
recursive-include lbry *.txt *.py
+20
View File
@@ -0,0 +1,20 @@
.PHONY: tools lint test idea
lint:
pylint --rcfile=setup.cfg lbry
#mypy --ignore-missing-imports lbry
test:
tox
idea:
mkdir -p .idea
cp -r scripts/idea/* .idea
start:
dropdb lbry --if-exists
createdb lbry
lbrynet start node \
--db-url=postgresql:///lbry --workers=0 --console=advanced --no-spv-address-filters \
--lbrycrd-rpc-user=lbry --lbrycrd-rpc-pass=somethingelse \
--lbrycrd-dir=${HOME}/.lbrycrd --data-dir=/tmp/tmp-lbrynet
+17 -13
View File
@@ -1,19 +1,19 @@
# <img src="https://raw.githubusercontent.com/lbryio/lbry/master/lbry.png" alt="LBRY" width="48" height="36" /> LBRY SDK [![Build Status](https://travis-ci.org/lbryio/lbry.svg?branch=master)](https://travis-ci.org/lbryio/lbry) [![Test Coverage](https://codecov.io/gh/lbryio/lbry/branch/master/graph/badge.svg)](https://codecov.io/gh/lbryio/lbry)
# <img src="https://raw.githubusercontent.com/lbryio/lbry-sdk/master/lbry.png" alt="LBRY" width="48" height="36" /> LBRY SDK [![Gitlab CI Badge](https://ci.lbry.tech/lbry/lbry-sdk/badges/master/pipeline.svg)](https://ci.lbry.tech/lbry/lbry-sdk)
LBRY is a decentralized peer-to-peer network providing distribution, discovery, and purchase of digital content (data). It utilizes the [LBRY blockchain](https://github.com/lbryio/lbrycrd) as a global namespace and database of digital content. Blockchain entries contain searchable content metadata, identities, rights and access rules. LBRY also provides a data network that consists of peers (seeders) uploading and downloading data from other peers, possibly in exchange for payments, as well as a distributed hash table used by peers to discover other peers.
LBRY is a decentralized peer-to-peer protocol for publishing and accessing digital content. It utilizes the [LBRY blockchain](https://github.com/lbryio/lbrycrd) as a global namespace and database of digital content. Blockchain entries contain searchable content metadata, identities, rights and access rules. LBRY also provides a data network that consists of peers (seeders) uploading and downloading data from other peers, possibly in exchange for payments, as well as a distributed hash table used by peers to discover other peers.
LBRY SDK for Python is currently the most full featured implementation of the LBRY Network protocols and includes many useful components and tools for building decentralized applications. Primary features and components:
LBRY SDK for Python is currently the most fully featured implementation of the LBRY Network protocols and includes many useful components and tools for building decentralized applications. Primary features and components include:
* Built on Python 3.7+ and `asyncio`.
* DHT (Distributed Hash Table) implementation for finding peers ([lbrynet.dht](https://github.com/lbryio/lbry/tree/master/lbrynet/dht)).
* Blob exchange protocol for downloading content and negotiating payments ([lbrynet.blob_exchange](https://github.com/lbryio/lbry/tree/master/lbrynet/blob_exchange)).
* Protobuf schema for encoding and decoding metadata stored on the blockchain ([lbrynet.schema](https://github.com/lbryio/lbry/tree/master/lbrynet/schema)).
* Wallet implementation for the LBRY blockchain ([lbrynet.extras.wallet](https://github.com/lbryio/lbry/tree/master/lbrynet/extras/wallet)).
* Daemon with a JSON-RPC API to ease building end user applications in any language and for automating various tasks ([lbrynet.extras.daemon](https://github.com/lbryio/lbry/tree/master/lbrynet/extras/daemon)).
* Kademlia DHT (Distributed Hash Table) implementation for finding peers to download from and announcing to peers what we have to host ([lbry.dht](https://github.com/lbryio/lbry-sdk/tree/master/lbry/dht)).
* Blob exchange protocol for transferring encrypted blobs of content and negotiating payments ([lbry.blob_exchange](https://github.com/lbryio/lbry-sdk/tree/master/lbry/blob_exchange)).
* Protobuf schema for encoding and decoding metadata stored on the blockchain ([lbry.schema](https://github.com/lbryio/lbry-sdk/tree/master/lbry/schema)).
* Wallet implementation for the LBRY blockchain ([lbry.wallet](https://github.com/lbryio/lbry-sdk/tree/master/lbry/wallet)).
* Daemon with a JSON-RPC API to ease building end user applications in any language and for automating various tasks ([lbry.extras.daemon](https://github.com/lbryio/lbry-sdk/tree/master/lbry/extras/daemon)).
## Installation
Our [releases page](https://github.com/lbryio/lbry/releases) contains pre-built binaries of the latest release, pre-releases, and past releases for macOS, Debian-based Linux, and Windows. [Automated travis builds](http://build.lbry.io/daemon/) are also available for testing.
Our [releases page](https://github.com/lbryio/lbry-sdk/releases) contains pre-built binaries of the latest release, pre-releases, and past releases for macOS, Debian-based Linux, and Windows. [Automated travis builds](http://build.lbry.io/daemon/) are also available for testing.
## Usage
@@ -27,13 +27,17 @@ With the daemon running, `lbrynet commands` will show you a list of commands.
The full API is documented [here](https://lbry.tech/api/sdk).
## Recommended hardware
The minimum hardware for a full node is 16cpus, 92gb of RAM, and 160gb of NVMe storage. The recommended hardware is 32cpus, 128gb of RAM, and 160gb of NVMe storage.
## Running from source
Installing from source is also relatively painless. Full instructions are in [INSTALL.md](INSTALL.md)
## Contributing
Contributions to this project are welcome, encouraged, and compensated. For more details, please check [this](https://lbry.io/faq/contributing) link.
Contributions to this project are welcome, encouraged, and compensated. For more details, please check [this](https://lbry.tech/contribute) link.
## License
@@ -41,11 +45,11 @@ This project is MIT licensed. For the full license, see [LICENSE](LICENSE).
## Security
We take security seriously. Please contact security@lbry.io regarding any security issues. [Our GPG key is here](https://lbry.io/faq/gpg-key) if you need it.
We take security seriously. Please contact security@lbry.com regarding any security issues. [Our GPG key is here](https://lbry.com/faq/gpg-key) if you need it.
## Contact
The primary contact for this project is [@eukreign](mailto:lex@lbry.io).
The primary contact for this project is [@eukreign](mailto:lex@lbry.com).
## Additional information and links
@@ -53,4 +57,4 @@ The documentation for the API can be found [here](https://lbry.tech/api/sdk).
Daemon defaults, ports, and other settings are documented [here](https://lbry.tech/resources/daemon-settings).
Settings can be configured using a daemon-settings.yml file. An example can be found [here](https://github.com/lbryio/lbry/blob/master/example_daemon_settings.yml).
Settings can be configured using a daemon-settings.yml file. An example can be found [here](https://github.com/lbryio/lbry-sdk/blob/master/example_daemon_settings.yml).
+5
View File
@@ -0,0 +1,5 @@
FROM ubuntu:20.04
COPY lbrynet /bin
RUN lbrynet --version
ENTRYPOINT ["lbrynet"]
CMD ["start", "node"]
+8
View File
@@ -0,0 +1,8 @@
FROM ubuntu:20.04
RUN apt-get update && \
apt-get install -y wget unzip && \
wget -nv https://build.lbry.io/lbrycrd/block_info_fix_try2/lbrycrd-linux.zip && \
unzip -d /bin lbrycrd-linux.zip && \
rm -rf lbrycrd-linux.zip /var/lib/apt/lists/*
RUN lbrycrdd --version
ENTRYPOINT ["lbrycrdd"]
+41
View File
@@ -0,0 +1,41 @@
version: "3.8"
volumes:
lbrycrd-data:
services:
postgres:
image: postgres:12
environment:
POSTGRES_USER: lbry
POSTGRES_PASSWORD: lbry
lbrycrd:
image: lbry/lbrycrd
build:
context: .
dockerfile: Dockerfile.lbrycrd
volumes:
- lbrycrd-data:/root/.lbrycrd
command: >
-rpcbind=lbrycrd
-rpcallowip=0.0.0.0/0
-rpcuser=lbryuser
-rpcpassword=lbrypass
-zmqpubhashblock=tcp://lbrycrd:29000
lbrynet:
image: lbry/lbrynet:fast_wallet_server_sync
depends_on:
- postgres
- lbrycrd
volumes:
- lbrycrd-data:/lbrycrd
command: >
start
--full-node
--api=0.0.0.0:5279
--db-url=postgresql://lbry:lbry@postgres:5432/lbry
--workers=12
--console=basic
--no-spv-address-filters
--lbrycrd-rpc-host=lbrycrd
--lbrycrd-rpc-user=lbryuser
--lbrycrd-rpc-pass=lbrypass
--lbrycrd-dir=/lbrycrd
+42
View File
@@ -0,0 +1,42 @@
import sys
import os
import re
import logging
import lbry.build_info as build_info_mod
log = logging.getLogger()
log.addHandler(logging.StreamHandler())
log.setLevel(logging.DEBUG)
def _check_and_set(d: dict, key: str, value: str):
try:
d[key]
except KeyError:
raise Exception(f"{key} var does not exist in {build_info_mod.__file__}")
d[key] = value
def main():
build_info = {item: build_info_mod.__dict__[item] for item in dir(build_info_mod) if not item.startswith("__")}
commit_hash = os.getenv('DOCKER_COMMIT', os.getenv('CI_COMMIT_SHA', os.getenv('TRAVIS_COMMIT')))
if commit_hash is None:
raise ValueError("Commit hash not found in env vars")
_check_and_set(build_info, "COMMIT_HASH", commit_hash[:6])
docker_tag = os.getenv('DOCKER_TAG')
if docker_tag:
_check_and_set(build_info, "DOCKER_TAG", docker_tag)
_check_and_set(build_info, "BUILD", "docker")
else:
ci_tag = os.getenv('CI_COMMIT_TAG', os.getenv('TRAVIS_TAG'))
_check_and_set(build_info, "BUILD", "release" if re.match(r'v\d+\.\d+\.\d+$', str(ci_tag)) else "qa")
log.debug("build info: %s", ", ".join([f"{k}={v}" for k, v in build_info.items()]))
with open(build_info_mod.__file__, 'w') as f:
f.write("\n".join([f"{k} = \"{v}\"" for k, v in build_info.items()]) + "\n")
if __name__ == '__main__':
sys.exit(main())
-307
View File
@@ -1,307 +0,0 @@
<!DOCTYPE html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="lang:clipboard.copy" content="Copy to clipboard">
<meta name="lang:clipboard.copied" content="Copied to clipboard">
<meta name="lang:search.language" content="en">
<meta name="lang:search.pipeline.stopwords" content="True">
<meta name="lang:search.pipeline.trimmer" content="True">
<meta name="lang:search.result.none" content="No matching documents">
<meta name="lang:search.result.one" content="1 matching document">
<meta name="lang:search.result.other" content="# matching documents">
<meta name="lang:search.tokenizer" content="[\s\-]+">
<link rel="shortcut icon" href="/assets/images/favicon.png">
<meta name="generator" content="mkdocs-0.17.3, mkdocs-material-2.7.0">
<title>LBRY</title>
<link rel="stylesheet" href="/assets/stylesheets/application.78aab2dc.css">
<link rel="stylesheet" href="/assets/stylesheets/application-palette.6079476c.css">
<script src="/assets/javascripts/modernizr.1aa3b519.js"></script>
<link href="https://fonts.gstatic.com" rel="preconnect" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,400i,700|Roboto+Mono">
<style>body,input{font-family:"Roboto","Helvetica Neue",Helvetica,Arial,sans-serif}code,kbd,pre{font-family:"Roboto Mono","Courier New",Courier,monospace}</style>
<link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons">
</head>
<body dir="ltr" data-md-color-primary="teal" data-md-color-accent="green">
<svg class="md-svg">
<defs>
<svg xmlns="http://www.w3.org/2000/svg" width="416" height="448"
viewBox="0 0 416 448" id="github">
<path fill="currentColor" d="M160 304q0 10-3.125 20.5t-10.75 19-18.125
8.5-18.125-8.5-10.75-19-3.125-20.5 3.125-20.5 10.75-19 18.125-8.5
18.125 8.5 10.75 19 3.125 20.5zM320 304q0 10-3.125 20.5t-10.75
19-18.125 8.5-18.125-8.5-10.75-19-3.125-20.5 3.125-20.5 10.75-19
18.125-8.5 18.125 8.5 10.75 19 3.125 20.5zM360
304q0-30-17.25-51t-46.75-21q-10.25 0-48.75 5.25-17.75 2.75-39.25
2.75t-39.25-2.75q-38-5.25-48.75-5.25-29.5 0-46.75 21t-17.25 51q0 22 8
38.375t20.25 25.75 30.5 15 35 7.375 37.25 1.75h42q20.5 0
37.25-1.75t35-7.375 30.5-15 20.25-25.75 8-38.375zM416 260q0 51.75-15.25
82.75-9.5 19.25-26.375 33.25t-35.25 21.5-42.5 11.875-42.875 5.5-41.75
1.125q-19.5 0-35.5-0.75t-36.875-3.125-38.125-7.5-34.25-12.875-30.25-20.25-21.5-28.75q-15.5-30.75-15.5-82.75
0-59.25 34-99-6.75-20.5-6.75-42.5 0-29 12.75-54.5 27 0 47.5 9.875t47.25
30.875q36.75-8.75 77.25-8.75 37 0 70 8 26.25-20.5
46.75-30.25t47.25-9.75q12.75 25.5 12.75 54.5 0 21.75-6.75 42 34 40 34
99.5z" />
</svg>
</defs>
</svg>
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="drawer">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="search">
<label class="md-overlay" data-md-component="overlay" for="drawer"></label>
<header class="md-header" data-md-component="header">
<nav class="md-header-nav md-grid">
<div class="md-flex">
<div class="md-flex__cell md-flex__cell--shrink">
<a href="/" title="LBRY" class="md-header-nav__button md-logo">
<img src="https://s3.amazonaws.com/files.lbry.io/logo-square-white-bookonly.png" alt="LBRY logo" width="24" height="24">
</a>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<label class="md-icon md-icon--menu md-header-nav__button" for="drawer"></label>
</div>
<div class="md-flex__cell md-flex__cell--stretch">
<div class="md-flex__ellipsis md-header-nav__title" data-md-component="title">
<span class="md-header-nav__topic">
LBRY
</span>
<span class="md-header-nav__topic">
</span>
</div>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<label class="md-icon md-icon--search md-header-nav__button" for="search"></label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="query" data-md-state="active">
<label class="md-icon md-search__icon" for="search"></label>
<button type="reset" class="md-icon md-search__icon" data-md-component="reset" tabindex="-1">
&#xE5CD;
</button>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" data-md-scrollfix>
<div class="md-search-result" data-md-component="result">
<div class="md-search-result__meta">
Type to start searching
</div>
<ol class="md-search-result__list"></ol>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<div class="md-header-nav__source">
<a href="https://github.com/lbryio/lbry/" title="Go to repository" class="md-source" data-md-source="github">
<div class="md-source__icon">
<svg viewBox="0 0 24 24" width="24" height="24">
<use xlink:href="#github" width="24" height="24"></use>
</svg>
</div>
<div class="md-source__repository">
GitHub
</div>
</a>
</div>
</div>
</div>
</nav>
</header>
<div class="md-container">
<main class="md-main">
<div class="md-main__inner md-grid" data-md-component="container">
<div class="md-sidebar md-sidebar--primary" data-md-component="navigation">
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" data-md-level="0">
<label class="md-nav__title md-nav__title--site" for="drawer">
<span class="md-nav__button md-logo">
<img src="https://s3.amazonaws.com/files.lbry.io/logo-square-white-bookonly.png" alt="LBRY logo" width="48" height="48">
</span>
LBRY
</label>
<div class="md-nav__source">
<a href="https://github.com/lbryio/lbry/" title="Go to repository" class="md-source" data-md-source="github">
<div class="md-source__icon">
<svg viewBox="0 0 24 24" width="24" height="24">
<use xlink:href="#github" width="24" height="24"></use>
</svg>
</div>
<div class="md-source__repository">
GitHub
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/" title="API" class="md-nav__link">
API
</a>
</li>
<li class="md-nav__item">
<a href="/cli/" title="CLI" class="md-nav__link">
CLI
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content">
<article class="md-content__inner md-typeset">
<h1>404 - Not found</h1>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-footer-copyright">
powered by
<a href="http://www.mkdocs.org">MkDocs</a>
and
<a href="https://squidfunk.github.io/mkdocs-material/">
Material for MkDocs</a>
</div>
</div>
</div>
</footer>
</div>
<script src="/assets/javascripts/application.8eb9be28.js"></script>
<script>app.initialize({version:"0.17.3",url:{base:""}})</script>
<script>!function(e,a,t,n,o,c,i){e.GoogleAnalyticsObject=o,e.ga=e.ga||function(){(e.ga.q=e.ga.q||[]).push(arguments)},e.ga.l=1*new Date,c=a.createElement(t),i=a.getElementsByTagName(t)[0],c.async=1,c.src="https://www.google-analytics.com/analytics.js",i.parentNode.insertBefore(c,i)}(window,document,"script",0,"ga"),ga("create","UA-60403362-1","auto"),ga("set","anonymizeIp",!0),ga("send","pageview");var links=document.getElementsByTagName("a");if(Array.prototype.map.call(links,function(e){e.host!=document.location.host&&e.addEventListener("click",function(){var a=e.getAttribute("data-md-action")||"follow";ga("send","event","outbound",a,e.href)})}),document.forms.search){var query=document.forms.search.query;query.addEventListener("blur",function(){if(this.value){var e=document.location.pathname;ga("send","pageview",e+"?q="+this.value)}})}</script>
</body>
</html>
+4892 -1512
View File
File diff suppressed because it is too large. Load diff
Binary file not shown.

Before

Width:  |  Height:  |  Size: 521 B

@@ -1,20 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="352" height="448"
viewBox="0 0 352 448" id="bitbucket">
<path fill="currentColor" d="M203.75 214.75q2 15.75-12.625 25.25t-27.875
1.5q-9.75-4.25-13.375-14.5t-0.125-20.5 13-14.5q9-4.5 18.125-3t16 8.875
6.875 16.875zM231.5 209.5q-3.5-26.75-28.25-41t-49.25-3.25q-15.75
7-25.125 22.125t-8.625 32.375q1 22.75 19.375 38.75t41.375 14q22.75-2
38-21t12.5-42zM291.25
74q-5-6.75-14-11.125t-14.5-5.5-17.75-3.125q-72.75-11.75-141.5 0.5-10.75
1.75-16.5 3t-13.75 5.5-12.5 10.75q7.5 7 19 11.375t18.375 5.5 21.875
2.875q57 7.25 112 0.25 15.75-2 22.375-3t18.125-5.375 18.75-11.625zM305.5
332.75q-2 6.5-3.875 19.125t-3.5 21-7.125 17.5-14.5 14.125q-21.5
12-47.375 17.875t-50.5 5.5-50.375-4.625q-11.5-2-20.375-4.5t-19.125-6.75-18.25-10.875-13-15.375q-6.25-24-14.25-73l1.5-4
4.5-2.25q55.75 37 126.625 37t126.875-37q5.25 1.5 6 5.75t-1.25 11.25-2
9.25zM350.75 92.5q-6.5 41.75-27.75 163.75-1.25 7.5-6.75 14t-10.875
10-13.625 7.75q-63 31.5-152.5
22-62-6.75-98.5-34.75-3.75-3-6.375-6.625t-4.25-8.75-2.25-8.5-1.5-9.875-1.375-8.75q-2.25-12.5-6.625-37.5t-7-40.375-5.875-36.875-5.5-39.5q0.75-6.5
4.375-12.125t7.875-9.375 11.25-7.5 11.5-5.625 12-4.625q31.25-11.5
78.25-16 94.75-9.25 169 12.5 38.75 11.5 53.75 30.5 4 5 4.125
12.75t-1.375 13.5z" />
</svg>

Before

Width:  |  Height:  |  Size: 1.4 KiB

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="416" height="448"
viewBox="0 0 416 448" id="github">
<path fill="currentColor" d="M160 304q0 10-3.125 20.5t-10.75 19-18.125
8.5-18.125-8.5-10.75-19-3.125-20.5 3.125-20.5 10.75-19 18.125-8.5
18.125 8.5 10.75 19 3.125 20.5zM320 304q0 10-3.125 20.5t-10.75
19-18.125 8.5-18.125-8.5-10.75-19-3.125-20.5 3.125-20.5 10.75-19
18.125-8.5 18.125 8.5 10.75 19 3.125 20.5zM360
304q0-30-17.25-51t-46.75-21q-10.25 0-48.75 5.25-17.75 2.75-39.25
2.75t-39.25-2.75q-38-5.25-48.75-5.25-29.5 0-46.75 21t-17.25 51q0 22 8
38.375t20.25 25.75 30.5 15 35 7.375 37.25 1.75h42q20.5 0
37.25-1.75t35-7.375 30.5-15 20.25-25.75 8-38.375zM416 260q0 51.75-15.25
82.75-9.5 19.25-26.375 33.25t-35.25 21.5-42.5 11.875-42.875 5.5-41.75
1.125q-19.5 0-35.5-0.75t-36.875-3.125-38.125-7.5-34.25-12.875-30.25-20.25-21.5-28.75q-15.5-30.75-15.5-82.75
0-59.25 34-99-6.75-20.5-6.75-42.5 0-29 12.75-54.5 27 0 47.5 9.875t47.25
30.875q36.75-8.75 77.25-8.75 37 0 70 8 26.25-20.5
46.75-30.25t47.25-9.75q12.75 25.5 12.75 54.5 0 21.75-6.75 42 34 40 34
99.5z" />
</svg>

Before

Width:  |  Height:  |  Size: 1.2 KiB

@@ -1,38 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="500" height="500"
viewBox="0 0 500 500" id="gitlab">
<g transform="translate(156.197863, 1.160267)">
<path fill="currentColor"
d="M93.667,473.347L93.667,473.347l90.684-279.097H2.983L93.667,
473.347L93.667,473.347z" />
</g>
<g transform="translate(28.531199, 1.160800)" opacity="0.7">
<path fill="currentColor"
d="M221.333,473.345L130.649,194.25H3.557L221.333,473.345L221.333,
473.345z" />
</g>
<g transform="translate(0.088533, 0.255867)" opacity="0.5">
<path fill="currentColor"
d="M32,195.155L32,195.155L4.441,279.97c-2.513,7.735,0.24,16.21,6.821,
20.99l238.514,173.29 L32,195.155L32,195.155z" />
</g>
<g transform="translate(29.421866, 280.255593)">
<path fill="currentColor"
d="M2.667-84.844h127.092L75.14-252.942c-2.811-8.649-15.047-8.649-17.856,
0L2.667-84.844 L2.667-84.844z" />
</g>
<g transform="translate(247.197860, 1.160800)" opacity="0.7">
<path fill="currentColor"
d="M2.667,473.345L93.351,194.25h127.092L2.667,473.345L2.667,
473.345z" />
</g>
<g transform="translate(246.307061, 0.255867)" opacity="0.5">
<path fill="currentColor"
d="M221.334,195.155L221.334,195.155l27.559,84.815c2.514,7.735-0.24,
16.21-6.821,20.99 L3.557,474.25L221.334,195.155L221.334,195.155z" />
</g>
<g transform="translate(336.973725, 280.255593)">
<path fill="currentColor"
d="M130.667-84.844H3.575l54.618-168.098c2.811-8.649,15.047-8.649,
17.856,0L130.667-84.844 L130.667-84.844z" />
</g>
</svg>

Before

Width:  |  Height:  |  Size: 1.6 KiB

@@ -1 +0,0 @@
!function(e,t){for(var n in t)e[n]=t[n]}(window,function(e){function t(r){if(n[r])return n[r].exports;var i=n[r]={i:r,l:!1,exports:{}};return e[r].call(i.exports,i,i.exports,t),i.l=!0,i.exports}var n={};return t.m=e,t.c=n,t.d=function(e,n,r){t.o(e,n)||Object.defineProperty(e,n,{configurable:!1,enumerable:!0,get:r})},t.n=function(e){var n=e&&e.__esModule?function(){return e.default}:function(){return e};return t.d(n,"a",n),n},t.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)},t.p="",t(t.s=6)}([function(e,t,n){"use strict";t.__esModule=!0,t.default={createElement:function(e,t){var n=document.createElement(e);t&&Array.prototype.forEach.call(Object.keys(t),function(e){n.setAttribute(e,t[e])});for(var r=arguments.length,i=Array(r>2?r-2:0),o=2;o<r;o++)i[o-2]=arguments[o];return function e(t){Array.prototype.forEach.call(t,function(t){"string"==typeof t||"number"==typeof t?n.textContent+=t:Array.isArray(t)?e(t):void 0!==t.__html?n.innerHTML+=t.__html:t instanceof Node&&n.appendChild(t)})}(i),n}},e.exports=t.default},function(e,t){var n;n=function(){return this}();try{n=n||Function("return this")()||(0,eval)("this")}catch(e){"object"==typeof window&&(n=window)}e.exports=n},function(e,t,n){"use strict";Object.defineProperty(t,"__esModule",{value:!0});var r="function"==typeof fetch?fetch.bind():function(e,t){return t=t||{},new Promise(function(n,r){function i(){var e,t=[],n=[],r={};return o.getAllResponseHeaders().replace(/^(.*?):\s*([\s\S]*?)$/gm,function(i,o,a){t.push(o=o.toLowerCase()),n.push([o,a]),e=r[o],r[o]=e?e+","+a:a}),{ok:1==(o.status/200|0),status:o.status,statusText:o.statusText,url:o.responseURL,clone:i,text:function(){return Promise.resolve(o.responseText)},json:function(){return Promise.resolve(o.responseText).then(JSON.parse)},blob:function(){return Promise.resolve(new Blob([o.response]))},headers:{keys:function(){return t},entries:function(){return n},get:function(e){return r[e.toLowerCase()]},has:function(e){return e.toLowerCase()in r}}}}var o=new XMLHttpRequest;o.open(t.method||"get",e);for(var a in t.headers)o.setRequestHeader(a,t.headers[a]);o.withCredentials="include"==t.credentials,o.onload=function(){n(i())},o.onerror=r,o.send(t.body)})};t.default=r},function(e,t,n){"use strict";function r(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}t.__esModule=!0;var i=function(){function e(t,n,i){var o=this;r(this,e),this.els_=Array.prototype.slice.call("string"==typeof t?document.querySelectorAll(t):[].concat(t)),this.handler_="function"==typeof i?{update:i}:i,this.events_=[].concat(n),this.update_=function(e){return o.handler_.update(e)}}return e.prototype.listen=function(){var e=this;this.els_.forEach(function(t){e.events_.forEach(function(n){t.addEventListener(n,e.update_,!1)})}),"function"==typeof this.handler_.setup&&this.handler_.setup()},e.prototype.unlisten=function(){var e=this;this.els_.forEach(function(t){e.events_.forEach(function(n){t.removeEventListener(n,e.update_)})}),"function"==typeof this.handler_.reset&&this.handler_.reset()},e}();t.default=i},,,function(e,t,n){"use strict";(function(e){function r(e){return e&&e.__esModule?e:{default:e}}function i(t){new f.default.Event.Listener(document,"DOMContentLoaded",function(){if(!(document.body instanceof HTMLElement))throw new ReferenceError;l.default.attach(document.body),Modernizr.addTest("ios",function(){return!!navigator.userAgent.match(/(iPad|iPhone|iPod)/g)});var t=document.querySelectorAll("table:not([class])");if(Array.prototype.forEach.call(t,function(t){var n=e.createElement("div",{class:"md-typeset__scrollwrap"},e.createElement("div",{class:"md-typeset__table"}));t.nextSibling?t.parentNode.insertBefore(n,t.nextSibling):t.parentNode.appendChild(n),n.children[0].appendChild(t)}),c.default.isSupported()){var n=document.querySelectorAll(".codehilite > pre, pre > code");Array.prototype.forEach.call(n,function(t,n){var r="__code_"+n,i=e.createElement("button",{class:"md-clipboard",title:h("clipboard.copy"),"data-clipboard-target":"#"+r+" pre, #"+r+" code"},e.createElement("span",{class:"md-clipboard__message"})),o=t.parentNode;o.id=r,o.insertBefore(i,t)});new c.default(".md-clipboard").on("success",function(e){var t=e.trigger.querySelector(".md-clipboard__message");if(!(t instanceof HTMLElement))throw new ReferenceError;e.clearSelection(),t.dataset.mdTimer&&clearTimeout(parseInt(t.dataset.mdTimer,10)),t.classList.add("md-clipboard__message--active"),t.innerHTML=h("clipboard.copied"),t.dataset.mdTimer=setTimeout(function(){t.classList.remove("md-clipboard__message--active"),t.dataset.mdTimer=""},2e3).toString()})}if(!Modernizr.details){var r=document.querySelectorAll("details > summary");Array.prototype.forEach.call(r,function(e){e.addEventListener("click",function(e){var t=e.target.parentNode;t.hasAttribute("open")?t.removeAttribute("open"):t.setAttribute("open","")})})}var i=function(){if(document.location.hash){var e=document.getElementById(document.location.hash.substring(1));if(!Line truncated
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,i,n;e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=(r=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=new function(){var e,n,t,s=[new r("hed",-1,1),new r("ethed",0,1),new r("ered",-1,1),new r("e",-1,1),new r("erede",3,1),new r("ende",3,1),new r("erende",5,1),new r("ene",3,1),new r("erne",3,1),new r("ere",3,1),new r("en",-1,1),new r("heden",10,1),new r("eren",10,1),new r("er",-1,1),new r("heder",13,1),new r("erer",13,1),new r("s",-1,2),new r("heds",16,1),new r("es",16,1),new r("endes",18,1),new r("erendes",19,1),new r("enes",18,1),new r("ernes",18,1),new r("eres",18,1),new r("ens",16,1),new r("hedens",24,1),new r("erens",24,1),new r("ers",16,1),new r("ets",16,1),new r("erets",28,1),new r("et",-1,1),new r("eret",30,1)],o=[new r("gd",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1)],a=[new r("ig",-1,1),new r("lig",0,1),new r("elig",1,1),new r("els",-1,1),new r("løst",-1,2)],d=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],u=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],c=new i;function l(){var e,r=c.limit-c.cursor;c.cursor>=n&&(e=c.limit_backward,c.limit_backward=n,c.ket=c.cursor,c.find_among_b(o,4)?(c.bra=c.cursor,c.limit_backward=e,c.cursor=c.limit-r,c.cursor>c.limit_backward&&(c.cursor--,c.bra=c.cursor,c.slice_del())):c.limit_backward=e)}this.setCurrent=function(e){c.setCurrent(e)},this.getCurrent=function(){return c.getCurrent()},this.stem=function(){var r,i=c.cursor;return function(){var r,i=c.cursor+3;if(n=c.limit,0<=i&&i<=c.limit){for(e=i;;){if(r=c.cursor,c.in_grouping(d,97,248)){c.cursor=r;break}if(c.cursor=r,r>=c.limit)return;c.cursor++}for(;!c.out_grouping(d,97,248);){if(c.cursor>=c.limit)return;c.cursor++}(n=c.cursor)<e&&(n=e)}}(),c.limit_backward=i,c.cursor=c.limit,function(){var e,r;if(c.cursor>=n&&(r=c.limit_backward,c.limit_backward=n,c.ket=c.cursor,e=c.find_among_b(s,32),c.limit_backward=r,e))switch(c.bra=c.cursor,e){case 1:c.slice_del();break;case 2:c.in_grouping_b(u,97,229)&&c.slice_del()}}(),c.cursor=c.limit,l(),c.cursor=c.limit,function(){var e,r,i,t=c.limit-c.cursor;if(c.ket=c.cursor,c.eq_s_b(2,"st")&&(c.bra=c.cursor,c.eq_s_b(2,"ig")&&c.slice_del()),c.cursor=c.limit-t,c.cursor>=n&&(r=c.limit_backward,c.limit_backward=n,c.ket=c.cursor,e=c.find_among_b(a,5),c.limit_backward=r,e))switch(c.bra=c.cursor,e){case 1:c.slice_del(),i=c.limit-c.cursor,l(),c.cursor=c.limit-i;break;case 2:c.slice_from("løs")}}(),c.cursor=c.limit,c.cursor>=n&&(r=c.limit_backward,c.limit_backward=n,c.ket=c.cursor,c.out_grouping_b(d,97,248)?(c.bra=c.cursor,t=c.slice_to(t),c.limit_backward=r,c.eq_v_b(t)&&c.slice_del()):c.limit_backward=r),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,n,i;e.de=function(){this.pipeline.reset(),this.pipeline.add(e.de.trimmer,e.de.stopWordFilter,e.de.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.de.stemmer))},e.de.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.de.trimmer=e.trimmerSupport.generateTrimmer(e.de.wordCharacters),e.Pipeline.registerFunction(e.de.trimmer,"trimmer-de"),e.de.stemmer=(r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=new function(){var e,i,s,t=[new r("",-1,6),new r("U",0,2),new r("Y",0,1),new r("ä",0,3),new r("ö",0,4),new r("ü",0,5)],o=[new r("e",-1,2),new r("em",-1,1),new r("en",-1,2),new r("ern",-1,1),new r("er",-1,1),new r("s",-1,3),new r("es",5,2)],c=[new r("en",-1,1),new r("er",-1,1),new r("st",-1,2),new r("est",2,1)],u=[new r("ig",-1,1),new r("lich",-1,1)],a=[new r("end",-1,1),new r("ig",-1,2),new r("ung",-1,1),new r("lich",-1,3),new r("isch",-1,2),new r("ik",-1,2),new r("heit",-1,3),new r("keit",-1,4)],d=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,8,0,32,8],l=[117,30,5],m=[117,30,4],h=new n;function w(e,r,n){return!(!h.eq_s(1,e)||(h.ket=h.cursor,!h.in_grouping(d,97,252)))&&(h.slice_from(r),h.cursor=n,!0)}function f(){for(;!h.in_grouping(d,97,252);){if(h.cursor>=h.limit)return!0;h.cursor++}for(;!h.out_grouping(d,97,252);){if(h.cursor>=h.limit)return!0;h.cursor++}return!1}function b(){return s<=h.cursor}function _(){return i<=h.cursor}this.setCurrent=function(e){h.setCurrent(e)},this.getCurrent=function(){return h.getCurrent()},this.stem=function(){var r=h.cursor;return function(){for(var e,r,n,i,s=h.cursor;;)if(e=h.cursor,h.bra=e,h.eq_s(1,"ß"))h.ket=h.cursor,h.slice_from("ss");else{if(e>=h.limit)break;h.cursor=e+1}for(h.cursor=s;;)for(r=h.cursor;;){if(n=h.cursor,h.in_grouping(d,97,252)){if(i=h.cursor,h.bra=i,w("u","U",n))break;if(h.cursor=i,w("y","Y",n))break}if(n>=h.limit)return void(h.cursor=r);h.cursor=n+1}}(),h.cursor=r,function(){s=h.limit,i=s;var r=h.cursor+3;0<=r&&r<=h.limit&&(e=r,f()||((s=h.cursor)<e&&(s=e),f()||(i=h.cursor)))}(),h.limit_backward=r,h.cursor=h.limit,function(){var e,r,n,i,s=h.limit-h.cursor;if(h.ket=h.cursor,(e=h.find_among_b(o,7))&&(h.bra=h.cursor,b()))switch(e){case 1:h.slice_del();break;case 2:h.slice_del(),h.ket=h.cursor,h.eq_s_b(1,"s")&&(h.bra=h.cursor,h.eq_s_b(3,"nis")&&h.slice_del());break;case 3:h.in_grouping_b(l,98,116)&&h.slice_del()}if(h.cursor=h.limit-s,h.ket=h.cursor,(e=h.find_among_b(c,4))&&(h.bra=h.cursor,b()))switch(e){case 1:h.slice_del();break;case 2:if(h.in_grouping_b(m,98,116)){var t=h.cursor-3;h.limit_backward<=t&&t<=h.limit&&(h.cursor=t,h.slice_del())}}if(h.cursor=h.limit-s,h.ket=h.cursor,(e=h.find_among_b(a,8))&&(h.bra=h.cursor,_()))switch(e){case 1:h.slice_del(),h.ket=h.cursor,h.eq_s_b(2,"ig")&&(h.bra=h.cursor,r=h.limit-h.cursor,h.eq_s_b(1,"e")||(h.cursor=h.limit-r,_()&&h.slice_del()));break;case 2:n=h.limit-h.cursor,h.eq_s_b(1,"e")||(h.cursor=h.limit-n,h.slice_del());break;case 3:if(h.slice_del(),h.ket=h.cursor,i=h.limit-h.cursor,!h.eq_s_b(2,"er")&&(h.cursor=h.limit-i,!h.eq_s_b(2,"en")))break;h.bra=h.cursor,b()&&h.slice_del();break;case 4:h.slice_del(),h.ket=h.cursor,(e=h.find_among_b(u,2))&&(h.bra=h.cursor,_()&&1==e&&h.slice_del())}}(),h.cursor=h.limit_backward,function(){for(var e,r;;){if(r=h.cursor,h.bra=r,!(e=h.find_among(t,6)))return;switch(h.ket=h.cursor,e){case 1:h.slice_from("y");break;case 2:case 5:h.slice_from("u");break;case 3:h.slice_from("a");break;case 4:h.slice_from("o");break;case 6:if(h.cursor>=h.limit)return;h.cursor++}}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}),e.Pipeline.registerFunction(e.de.stemmer,"stemmer-de"),e.de.stopWordFilter=e.generateStopWordFilter("aber alle allem allen aller alles als also am an ander andere anderem anderen anderer anderes anderm andern anderr anders auch auf aus bei bin bis bist da damit dann das dasselbe dazu daß dein deine deinem deinen deiner deines dem demselben den denn denselben der derer derselbe derselben des desselben dessen dich die dies diese dieselbe dieselben diesem diesen dieser dieses dir doch dort du durch ein eine einem einen einer eines einig einige einigem einigen einiger einiges einmal er es etwas euch euer eure eurem euren eurer eures für gegen gewesen hab habe haben hat hatte hatten hier hin hinter ich ihm ihn ihnen ihr ihre ihrem ihren ihrer ihres im in indem ins ist jede jedem jedenLine truncated
-1
View File
@@ -1 +0,0 @@
!function(r,e){"function"==typeof define&&define.amd?define(e):"object"==typeof exports?module.exports=e():e()(r.lunr)}(this,function(){return function(r){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var e,i,n;r.du=function(){this.pipeline.reset(),this.pipeline.add(r.du.trimmer,r.du.stopWordFilter,r.du.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(r.du.stemmer))},r.du.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",r.du.trimmer=r.trimmerSupport.generateTrimmer(r.du.wordCharacters),r.Pipeline.registerFunction(r.du.trimmer,"trimmer-du"),r.du.stemmer=(e=r.stemmerSupport.Among,i=r.stemmerSupport.SnowballProgram,n=new function(){var r,n,o,t=[new e("",-1,6),new e("á",0,1),new e("ä",0,1),new e("é",0,2),new e("ë",0,2),new e("í",0,3),new e("ï",0,3),new e("ó",0,4),new e("ö",0,4),new e("ú",0,5),new e("ü",0,5)],s=[new e("",-1,3),new e("I",0,2),new e("Y",0,1)],u=[new e("dd",-1,-1),new e("kk",-1,-1),new e("tt",-1,-1)],c=[new e("ene",-1,2),new e("se",-1,3),new e("en",-1,2),new e("heden",2,1),new e("s",-1,3)],a=[new e("end",-1,1),new e("ig",-1,2),new e("ing",-1,1),new e("lijk",-1,3),new e("baar",-1,4),new e("bar",-1,5)],l=[new e("aa",-1,-1),new e("ee",-1,-1),new e("oo",-1,-1),new e("uu",-1,-1)],m=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],d=[1,0,0,17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],f=[17,67,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],_=new i;function w(r){return _.cursor=r,r>=_.limit||(_.cursor++,!1)}function b(){for(;!_.in_grouping(m,97,232);){if(_.cursor>=_.limit)return!0;_.cursor++}for(;!_.out_grouping(m,97,232);){if(_.cursor>=_.limit)return!0;_.cursor++}return!1}function p(){return n<=_.cursor}function g(){return r<=_.cursor}function h(){var r=_.limit-_.cursor;_.find_among_b(u,3)&&(_.cursor=_.limit-r,_.ket=_.cursor,_.cursor>_.limit_backward&&(_.cursor--,_.bra=_.cursor,_.slice_del()))}function k(){var r;o=!1,_.ket=_.cursor,_.eq_s_b(1,"e")&&(_.bra=_.cursor,p()&&(r=_.limit-_.cursor,_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-r,_.slice_del(),o=!0,h())))}function v(){var r;p()&&(r=_.limit-_.cursor,_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-r,_.eq_s_b(3,"gem")||(_.cursor=_.limit-r,_.slice_del(),h())))}this.setCurrent=function(r){_.setCurrent(r)},this.getCurrent=function(){return _.getCurrent()},this.stem=function(){var e=_.cursor;return function(){for(var r,e,i,n=_.cursor;;){if(_.bra=_.cursor,r=_.find_among(t,11))switch(_.ket=_.cursor,r){case 1:_.slice_from("a");continue;case 2:_.slice_from("e");continue;case 3:_.slice_from("i");continue;case 4:_.slice_from("o");continue;case 5:_.slice_from("u");continue;case 6:if(_.cursor>=_.limit)break;_.cursor++;continue}break}for(_.cursor=n,_.bra=n,_.eq_s(1,"y")?(_.ket=_.cursor,_.slice_from("Y")):_.cursor=n;;)if(e=_.cursor,_.in_grouping(m,97,232)){if(i=_.cursor,_.bra=i,_.eq_s(1,"i"))_.ket=_.cursor,_.in_grouping(m,97,232)&&(_.slice_from("I"),_.cursor=e);else if(_.cursor=i,_.eq_s(1,"y"))_.ket=_.cursor,_.slice_from("Y"),_.cursor=e;else if(w(e))break}else if(w(e))break}(),_.cursor=e,n=_.limit,r=n,b()||((n=_.cursor)<3&&(n=3),b()||(r=_.cursor)),_.limit_backward=e,_.cursor=_.limit,function(){var r,e,i,n,t,s,u=_.limit-_.cursor;if(_.ket=_.cursor,r=_.find_among_b(c,5))switch(_.bra=_.cursor,r){case 1:p()&&_.slice_from("heid");break;case 2:v();break;case 3:p()&&_.out_grouping_b(f,97,232)&&_.slice_del()}if(_.cursor=_.limit-u,k(),_.cursor=_.limit-u,_.ket=_.cursor,_.eq_s_b(4,"heid")&&(_.bra=_.cursor,g()&&(e=_.limit-_.cursor,_.eq_s_b(1,"c")||(_.cursor=_.limit-e,_.slice_del(),_.ket=_.cursor,_.eq_s_b(2,"en")&&(_.bra=_.cursor,v())))),_.cursor=_.limit-u,_.ket=_.cursor,r=_.find_among_b(a,6))switch(_.bra=_.cursor,r){case 1:if(g()){if(_.slice_del(),i=_.limit-_.cursor,_.ket=_.cursor,_.eq_s_b(2,"ig")&&(_.bra=_.cursor,g()&&(n=_.limit-_.cursor,!_.eq_s_b(1,"e")))){_.cursor=_.limit-n,_.slice_del();break}_.cursor=_.limit-i,h()}break;case 2:g()&&(t=_.limit-_.cursor,_.eq_s_b(1,"e")||(_.cursor=_.limit-t,_.slice_del()));break;case 3:g()&&(_.slice_del(),k());break;case 4:g()&&_.slice_del();break;case 5:g()&&o&&_.slice_del()}_.cursor=_.limit-u,_.out_grouping_b(d,73,232)&&(s=_.limit-_.cursor,_.find_among_b(l,4)&&_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-s,_.ket=_.cursor,_.cursor>_.limit_backward&&(_.cursor--,_.bra=_.cursor,_.slice_del())))}(),_.cursor=_.limit_backward,function(){for(var r;;)if(_.bra=_.cursor,r=_.find_among(s,3))switch(_.ket=_.cursor,r){case 1:_.slice_from("y");break;case 2:_.slice_from("i");break;case 3:if(_.cursor>=_.limit)return;_.cursor++}}(),!0}},function(r){return"function"==typeof r.update?r.update(function(r){return n.setCurrent(r),n.stem(),n.getCurrent()}):(n.setCurrent(r),n.stem(),n.getCurrent())}),r.PLine truncated
-1
View File
@@ -1 +0,0 @@
!function(e,s){"function"==typeof define&&define.amd?define(s):"object"==typeof exports?module.exports=s():s()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var s,r,n;e.es=function(){this.pipeline.reset(),this.pipeline.add(e.es.trimmer,e.es.stopWordFilter,e.es.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.es.stemmer))},e.es.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.es.trimmer=e.trimmerSupport.generateTrimmer(e.es.wordCharacters),e.Pipeline.registerFunction(e.es.trimmer,"trimmer-es"),e.es.stemmer=(s=e.stemmerSupport.Among,r=e.stemmerSupport.SnowballProgram,n=new function(){var e,n,i,a=[new s("",-1,6),new s("á",0,1),new s("é",0,2),new s("í",0,3),new s("ó",0,4),new s("ú",0,5)],t=[new s("la",-1,-1),new s("sela",0,-1),new s("le",-1,-1),new s("me",-1,-1),new s("se",-1,-1),new s("lo",-1,-1),new s("selo",5,-1),new s("las",-1,-1),new s("selas",7,-1),new s("les",-1,-1),new s("los",-1,-1),new s("selos",10,-1),new s("nos",-1,-1)],o=[new s("ando",-1,6),new s("iendo",-1,6),new s("yendo",-1,7),new s("ándo",-1,2),new s("iéndo",-1,1),new s("ar",-1,6),new s("er",-1,6),new s("ir",-1,6),new s("ár",-1,3),new s("ér",-1,4),new s("ír",-1,5)],u=[new s("ic",-1,-1),new s("ad",-1,-1),new s("os",-1,-1),new s("iv",-1,1)],w=[new s("able",-1,1),new s("ible",-1,1),new s("ante",-1,1)],c=[new s("ic",-1,1),new s("abil",-1,1),new s("iv",-1,1)],m=[new s("ica",-1,1),new s("ancia",-1,2),new s("encia",-1,5),new s("adora",-1,2),new s("osa",-1,1),new s("ista",-1,1),new s("iva",-1,9),new s("anza",-1,1),new s("logía",-1,3),new s("idad",-1,8),new s("able",-1,1),new s("ible",-1,1),new s("ante",-1,2),new s("mente",-1,7),new s("amente",13,6),new s("ación",-1,2),new s("ución",-1,4),new s("ico",-1,1),new s("ismo",-1,1),new s("oso",-1,1),new s("amiento",-1,1),new s("imiento",-1,1),new s("ivo",-1,9),new s("ador",-1,2),new s("icas",-1,1),new s("ancias",-1,2),new s("encias",-1,5),new s("adoras",-1,2),new s("osas",-1,1),new s("istas",-1,1),new s("ivas",-1,9),new s("anzas",-1,1),new s("logías",-1,3),new s("idades",-1,8),new s("ables",-1,1),new s("ibles",-1,1),new s("aciones",-1,2),new s("uciones",-1,4),new s("adores",-1,2),new s("antes",-1,2),new s("icos",-1,1),new s("ismos",-1,1),new s("osos",-1,1),new s("amientos",-1,1),new s("imientos",-1,1),new s("ivos",-1,9)],l=[new s("ya",-1,1),new s("ye",-1,1),new s("yan",-1,1),new s("yen",-1,1),new s("yeron",-1,1),new s("yendo",-1,1),new s("yo",-1,1),new s("yas",-1,1),new s("yes",-1,1),new s("yais",-1,1),new s("yamos",-1,1),new s("yó",-1,1)],d=[new s("aba",-1,2),new s("ada",-1,2),new s("ida",-1,2),new s("ara",-1,2),new s("iera",-1,2),new s("ía",-1,2),new s("aría",5,2),new s("ería",5,2),new s("iría",5,2),new s("ad",-1,2),new s("ed",-1,2),new s("id",-1,2),new s("ase",-1,2),new s("iese",-1,2),new s("aste",-1,2),new s("iste",-1,2),new s("an",-1,2),new s("aban",16,2),new s("aran",16,2),new s("ieran",16,2),new s("ían",16,2),new s("arían",20,2),new s("erían",20,2),new s("irían",20,2),new s("en",-1,1),new s("asen",24,2),new s("iesen",24,2),new s("aron",-1,2),new s("ieron",-1,2),new s("arán",-1,2),new s("erán",-1,2),new s("irán",-1,2),new s("ado",-1,2),new s("ido",-1,2),new s("ando",-1,2),new s("iendo",-1,2),new s("ar",-1,2),new s("er",-1,2),new s("ir",-1,2),new s("as",-1,2),new s("abas",39,2),new s("adas",39,2),new s("idas",39,2),new s("aras",39,2),new s("ieras",39,2),new s("ías",39,2),new s("arías",45,2),new s("erías",45,2),new s("irías",45,2),new s("es",-1,1),new s("ases",49,2),new s("ieses",49,2),new s("abais",-1,2),new s("arais",-1,2),new s("ierais",-1,2),new s("íais",-1,2),new s("aríais",55,2),new s("eríais",55,2),new s("iríais",55,2),new s("aseis",-1,2),new s("ieseis",-1,2),new s("asteis",-1,2),new s("isteis",-1,2),new s("áis",-1,2),new s("éis",-1,1),new s("aréis",64,2),new s("eréis",64,2),new s("iréis",64,2),new s("ados",-1,2),new s("idos",-1,2),new s("amos",-1,2),new s("ábamos",70,2),new s("áramos",70,2),new s("iéramos",70,2),new s("íamos",70,2),new s("aríamos",74,2),new s("eríamos",74,2),new s("iríamos",74,2),new s("emos",-1,1),new s("aremos",78,2),new s("eremos",78,2),new s("iremos",78,2),new s("ásemos",78,2),new s("iésemos",78,2),new s("imos",-1,2),new s("arás",-1,2),new s("erás",-1,2),new s("irás",-1,2),new s("ís",-1,2),new s("ará",-1,2),new s("erá",-1,2),new s("irá",-1,2),new s("aré",-1,2),new s("eré",-1,2),new s("iré",-1,2),new s("ió",-1,2)],b=[new s("a",-1,1),new s("e",-1,2),new s("o",-1,1),new s("os",-1,1),new s("á",-1,1),new s("é",-1,2),new s("í",-1,1),new s("ó",-1,1)],f=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,1,17,4,1Line truncated
-1
View File
@@ -1 +0,0 @@
!function(i,e){"function"==typeof define&&define.amd?define(e):"object"==typeof exports?module.exports=e():e()(i.lunr)}(this,function(){return function(i){if(void 0===i)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===i.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var e,r,n;i.fi=function(){this.pipeline.reset(),this.pipeline.add(i.fi.trimmer,i.fi.stopWordFilter,i.fi.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(i.fi.stemmer))},i.fi.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",i.fi.trimmer=i.trimmerSupport.generateTrimmer(i.fi.wordCharacters),i.Pipeline.registerFunction(i.fi.trimmer,"trimmer-fi"),i.fi.stemmer=(e=i.stemmerSupport.Among,r=i.stemmerSupport.SnowballProgram,n=new function(){var i,n,t,s,l=[new e("pa",-1,1),new e("sti",-1,2),new e("kaan",-1,1),new e("han",-1,1),new e("kin",-1,1),new e("hän",-1,1),new e("kään",-1,1),new e("ko",-1,1),new e("pä",-1,1),new e("kö",-1,1)],o=[new e("lla",-1,-1),new e("na",-1,-1),new e("ssa",-1,-1),new e("ta",-1,-1),new e("lta",3,-1),new e("sta",3,-1)],a=[new e("llä",-1,-1),new e("nä",-1,-1),new e("ssä",-1,-1),new e("tä",-1,-1),new e("ltä",3,-1),new e("stä",3,-1)],u=[new e("lle",-1,-1),new e("ine",-1,-1)],c=[new e("nsa",-1,3),new e("mme",-1,3),new e("nne",-1,3),new e("ni",-1,2),new e("si",-1,1),new e("an",-1,4),new e("en",-1,6),new e("än",-1,5),new e("nsä",-1,3)],m=[new e("aa",-1,-1),new e("ee",-1,-1),new e("ii",-1,-1),new e("oo",-1,-1),new e("uu",-1,-1),new e("ää",-1,-1),new e("öö",-1,-1)],w=[new e("a",-1,8),new e("lla",0,-1),new e("na",0,-1),new e("ssa",0,-1),new e("ta",0,-1),new e("lta",4,-1),new e("sta",4,-1),new e("tta",4,9),new e("lle",-1,-1),new e("ine",-1,-1),new e("ksi",-1,-1),new e("n",-1,7),new e("han",11,1),new e("den",11,-1,C),new e("seen",11,-1,v),new e("hen",11,2),new e("tten",11,-1,C),new e("hin",11,3),new e("siin",11,-1,C),new e("hon",11,4),new e("hän",11,5),new e("hön",11,6),new e("ä",-1,8),new e("llä",22,-1),new e("nä",22,-1),new e("ssä",22,-1),new e("tä",22,-1),new e("ltä",26,-1),new e("stä",26,-1),new e("ttä",26,9)],_=[new e("eja",-1,-1),new e("mma",-1,1),new e("imma",1,-1),new e("mpa",-1,1),new e("impa",3,-1),new e("mmi",-1,1),new e("immi",5,-1),new e("mpi",-1,1),new e("impi",7,-1),new e("ejä",-1,-1),new e("mmä",-1,1),new e("immä",10,-1),new e("mpä",-1,1),new e("impä",12,-1)],k=[new e("i",-1,-1),new e("j",-1,-1)],b=[new e("mma",-1,1),new e("imma",0,-1)],d=[17,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8],f=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,8,0,32],h=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,32],p=[17,97,24,1,0,0,0,0,0,0,0,0,0,0,0,0,8,0,32],g=new r;function j(){for(var i;i=g.cursor,!g.in_grouping(f,97,246);){if(g.cursor=i,i>=g.limit)return!0;g.cursor++}for(g.cursor=i;!g.out_grouping(f,97,246);){if(g.cursor>=g.limit)return!0;g.cursor++}return!1}function q(){var i,e;if(g.cursor>=s)if(e=g.limit_backward,g.limit_backward=s,g.ket=g.cursor,i=g.find_among_b(l,10)){switch(g.bra=g.cursor,g.limit_backward=e,i){case 1:if(!g.in_grouping_b(p,97,246))return;break;case 2:if(!(t<=g.cursor))return}g.slice_del()}else g.limit_backward=e}function v(){return g.find_among_b(m,7)}function C(){return g.eq_s_b(1,"i")&&g.in_grouping_b(h,97,246)}this.setCurrent=function(i){g.setCurrent(i)},this.getCurrent=function(){return g.getCurrent()},this.stem=function(){var e,r=g.cursor;return s=g.limit,t=s,j()||(s=g.cursor,j()||(t=g.cursor)),i=!1,g.limit_backward=r,g.cursor=g.limit,q(),g.cursor=g.limit,function(){var i,e,r;if(g.cursor>=s)if(e=g.limit_backward,g.limit_backward=s,g.ket=g.cursor,i=g.find_among_b(c,9))switch(g.bra=g.cursor,g.limit_backward=e,i){case 1:r=g.limit-g.cursor,g.eq_s_b(1,"k")||(g.cursor=g.limit-r,g.slice_del());break;case 2:g.slice_del(),g.ket=g.cursor,g.eq_s_b(3,"kse")&&(g.bra=g.cursor,g.slice_from("ksi"));break;case 3:g.slice_del();break;case 4:g.find_among_b(o,6)&&g.slice_del();break;case 5:g.find_among_b(a,6)&&g.slice_del();break;case 6:g.find_among_b(u,2)&&g.slice_del()}else g.limit_backward=e}(),g.cursor=g.limit,function(){var e,r,n;if(g.cursor>=s)if(r=g.limit_backward,g.limit_backward=s,g.ket=g.cursor,e=g.find_among_b(w,30)){switch(g.bra=g.cursor,g.limit_backward=r,e){case 1:if(!g.eq_s_b(1,"a"))return;break;case 2:case 9:if(!g.eq_s_b(1,"e"))return;break;case 3:if(!g.eq_s_b(1,"i"))return;break;case 4:if(!g.eq_s_b(1,"o"))return;break;case 5:if(!g.eq_s_b(1,"ä"))return;break;case 6:if(!g.eq_s_b(1,"ö"))return;break;case 7:if(n=g.limit-g.cursor,!v()&&(g.cursor=g.limit-n,!g.eq_s_b(2,"ie"))){g.cursor=g.limit-n;break}if(g.cursor=g.limit-n,g.cursor<=g.limit_backward){g.cursor=g.limit-n;break}g.cursor--,g.bra=g.cursor;break;case 8:if(!g.in_grouping_b(f,97,246)||!g.out_grouping_b(f,97,246))return}g.slice_del(),i=!0}else g.limiLine truncated
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,s,i;e.fr=function(){this.pipeline.reset(),this.pipeline.add(e.fr.trimmer,e.fr.stopWordFilter,e.fr.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.fr.stemmer))},e.fr.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.fr.trimmer=e.trimmerSupport.generateTrimmer(e.fr.wordCharacters),e.Pipeline.registerFunction(e.fr.trimmer,"trimmer-fr"),e.fr.stemmer=(r=e.stemmerSupport.Among,s=e.stemmerSupport.SnowballProgram,i=new function(){var e,i,n,t=[new r("col",-1,-1),new r("par",-1,-1),new r("tap",-1,-1)],u=[new r("",-1,4),new r("I",0,1),new r("U",0,2),new r("Y",0,3)],o=[new r("iqU",-1,3),new r("abl",-1,3),new r("Ièr",-1,4),new r("ièr",-1,4),new r("eus",-1,2),new r("iv",-1,1)],c=[new r("ic",-1,2),new r("abil",-1,1),new r("iv",-1,3)],a=[new r("iqUe",-1,1),new r("atrice",-1,2),new r("ance",-1,1),new r("ence",-1,5),new r("logie",-1,3),new r("able",-1,1),new r("isme",-1,1),new r("euse",-1,11),new r("iste",-1,1),new r("ive",-1,8),new r("if",-1,8),new r("usion",-1,4),new r("ation",-1,2),new r("ution",-1,4),new r("ateur",-1,2),new r("iqUes",-1,1),new r("atrices",-1,2),new r("ances",-1,1),new r("ences",-1,5),new r("logies",-1,3),new r("ables",-1,1),new r("ismes",-1,1),new r("euses",-1,11),new r("istes",-1,1),new r("ives",-1,8),new r("ifs",-1,8),new r("usions",-1,4),new r("ations",-1,2),new r("utions",-1,4),new r("ateurs",-1,2),new r("ments",-1,15),new r("ements",30,6),new r("issements",31,12),new r("ités",-1,7),new r("ment",-1,15),new r("ement",34,6),new r("issement",35,12),new r("amment",34,13),new r("emment",34,14),new r("aux",-1,10),new r("eaux",39,9),new r("eux",-1,1),new r("ité",-1,7)],l=[new r("ira",-1,1),new r("ie",-1,1),new r("isse",-1,1),new r("issante",-1,1),new r("i",-1,1),new r("irai",4,1),new r("ir",-1,1),new r("iras",-1,1),new r("ies",-1,1),new r("îmes",-1,1),new r("isses",-1,1),new r("issantes",-1,1),new r("îtes",-1,1),new r("is",-1,1),new r("irais",13,1),new r("issais",13,1),new r("irions",-1,1),new r("issions",-1,1),new r("irons",-1,1),new r("issons",-1,1),new r("issants",-1,1),new r("it",-1,1),new r("irait",21,1),new r("issait",21,1),new r("issant",-1,1),new r("iraIent",-1,1),new r("issaIent",-1,1),new r("irent",-1,1),new r("issent",-1,1),new r("iront",-1,1),new r("ît",-1,1),new r("iriez",-1,1),new r("issiez",-1,1),new r("irez",-1,1),new r("issez",-1,1)],w=[new r("a",-1,3),new r("era",0,2),new r("asse",-1,3),new r("ante",-1,3),new r("ée",-1,2),new r("ai",-1,3),new r("erai",5,2),new r("er",-1,2),new r("as",-1,3),new r("eras",8,2),new r("âmes",-1,3),new r("asses",-1,3),new r("antes",-1,3),new r("âtes",-1,3),new r("ées",-1,2),new r("ais",-1,3),new r("erais",15,2),new r("ions",-1,1),new r("erions",17,2),new r("assions",17,3),new r("erons",-1,2),new r("ants",-1,3),new r("és",-1,2),new r("ait",-1,3),new r("erait",23,2),new r("ant",-1,3),new r("aIent",-1,3),new r("eraIent",26,2),new r("èrent",-1,2),new r("assent",-1,3),new r("eront",-1,2),new r("ât",-1,3),new r("ez",-1,2),new r("iez",32,2),new r("eriez",33,2),new r("assiez",33,3),new r("erez",32,2),new r("é",-1,2)],f=[new r("e",-1,3),new r("Ière",0,2),new r("ière",0,2),new r("ion",-1,1),new r("Ier",-1,2),new r("ier",-1,2),new r("ë",-1,4)],m=[new r("ell",-1,-1),new r("eill",-1,-1),new r("enn",-1,-1),new r("onn",-1,-1),new r("ett",-1,-1)],_=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,128,130,103,8,5],b=[1,65,20,0,0,0,0,0,0,0,0,0,0,0,0,0,128],d=new s;function k(e,r,s){return!(!d.eq_s(1,e)||(d.ket=d.cursor,!d.in_grouping(_,97,251)))&&(d.slice_from(r),d.cursor=s,!0)}function p(e,r,s){return!!d.eq_s(1,e)&&(d.ket=d.cursor,d.slice_from(r),d.cursor=s,!0)}function g(){for(;!d.in_grouping(_,97,251);){if(d.cursor>=d.limit)return!0;d.cursor++}for(;!d.out_grouping(_,97,251);){if(d.cursor>=d.limit)return!0;d.cursor++}return!1}function q(){return n<=d.cursor}function v(){return i<=d.cursor}function h(){return e<=d.cursor}function z(){if(!function(){var e,r;if(d.ket=d.cursor,e=d.find_among_b(a,43)){switch(d.bra=d.cursor,e){case 1:if(!h())return!1;d.slice_del();break;case 2:if(!h())return!1;d.slice_del(),d.ket=d.cursor,d.eq_s_b(2,"ic")&&(d.bra=d.cursor,h()?d.slice_del():d.slice_from("iqU"));break;case 3:if(!h())return!1;d.slice_from("log");break;case 4:if(!h())return!1;d.slice_from("u");break;case 5:if(!h())return!1;d.slice_from("ent");break;case 6:if(!q())return!1;if(d.slice_del(),d.ket=d.cursor,e=d.find_among_b(o,6))switch(d.bra=d.cursor,e){case 1:h()&&(d.slice_del(),d.ket=d.cursor,d.eq_s_Line truncated
-1
View File
@@ -1 +0,0 @@
!function(e,n){"function"==typeof define&&define.amd?define(n):"object"==typeof exports?module.exports=n():n()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var n,r,i;e.hu=function(){this.pipeline.reset(),this.pipeline.add(e.hu.trimmer,e.hu.stopWordFilter,e.hu.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.hu.stemmer))},e.hu.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.hu.trimmer=e.trimmerSupport.generateTrimmer(e.hu.wordCharacters),e.Pipeline.registerFunction(e.hu.trimmer,"trimmer-hu"),e.hu.stemmer=(n=e.stemmerSupport.Among,r=e.stemmerSupport.SnowballProgram,i=new function(){var e,i=[new n("cs",-1,-1),new n("dzs",-1,-1),new n("gy",-1,-1),new n("ly",-1,-1),new n("ny",-1,-1),new n("sz",-1,-1),new n("ty",-1,-1),new n("zs",-1,-1)],a=[new n("á",-1,1),new n("é",-1,2)],t=[new n("bb",-1,-1),new n("cc",-1,-1),new n("dd",-1,-1),new n("ff",-1,-1),new n("gg",-1,-1),new n("jj",-1,-1),new n("kk",-1,-1),new n("ll",-1,-1),new n("mm",-1,-1),new n("nn",-1,-1),new n("pp",-1,-1),new n("rr",-1,-1),new n("ccs",-1,-1),new n("ss",-1,-1),new n("zzs",-1,-1),new n("tt",-1,-1),new n("vv",-1,-1),new n("ggy",-1,-1),new n("lly",-1,-1),new n("nny",-1,-1),new n("tty",-1,-1),new n("ssz",-1,-1),new n("zz",-1,-1)],s=[new n("al",-1,1),new n("el",-1,2)],c=[new n("ba",-1,-1),new n("ra",-1,-1),new n("be",-1,-1),new n("re",-1,-1),new n("ig",-1,-1),new n("nak",-1,-1),new n("nek",-1,-1),new n("val",-1,-1),new n("vel",-1,-1),new n("ul",-1,-1),new n("nál",-1,-1),new n("nél",-1,-1),new n("ból",-1,-1),new n("ról",-1,-1),new n("tól",-1,-1),new n("bõl",-1,-1),new n("rõl",-1,-1),new n("tõl",-1,-1),new n("ül",-1,-1),new n("n",-1,-1),new n("an",19,-1),new n("ban",20,-1),new n("en",19,-1),new n("ben",22,-1),new n("képpen",22,-1),new n("on",19,-1),new n("ön",19,-1),new n("képp",-1,-1),new n("kor",-1,-1),new n("t",-1,-1),new n("at",29,-1),new n("et",29,-1),new n("ként",29,-1),new n("anként",32,-1),new n("enként",32,-1),new n("onként",32,-1),new n("ot",29,-1),new n("ért",29,-1),new n("öt",29,-1),new n("hez",-1,-1),new n("hoz",-1,-1),new n("höz",-1,-1),new n("vá",-1,-1),new n("vé",-1,-1)],w=[new n("án",-1,2),new n("én",-1,1),new n("ánként",-1,3)],o=[new n("stul",-1,2),new n("astul",0,1),new n("ástul",0,3),new n("stül",-1,2),new n("estül",3,1),new n("éstül",3,4)],l=[new n("á",-1,1),new n("é",-1,2)],u=[new n("k",-1,7),new n("ak",0,4),new n("ek",0,6),new n("ok",0,5),new n("ák",0,1),new n("ék",0,2),new n("ök",0,3)],m=[new n("éi",-1,7),new n("áéi",0,6),new n("ééi",0,5),new n("é",-1,9),new n("ké",3,4),new n("aké",4,1),new n("eké",4,1),new n("oké",4,1),new n("áké",4,3),new n("éké",4,2),new n("öké",4,1),new n("éé",3,8)],k=[new n("a",-1,18),new n("ja",0,17),new n("d",-1,16),new n("ad",2,13),new n("ed",2,13),new n("od",2,13),new n("ád",2,14),new n("éd",2,15),new n("öd",2,13),new n("e",-1,18),new n("je",9,17),new n("nk",-1,4),new n("unk",11,1),new n("ánk",11,2),new n("énk",11,3),new n("ünk",11,1),new n("uk",-1,8),new n("juk",16,7),new n("ájuk",17,5),new n("ük",-1,8),new n("jük",19,7),new n("éjük",20,6),new n("m",-1,12),new n("am",22,9),new n("em",22,9),new n("om",22,9),new n("ám",22,10),new n("ém",22,11),new n("o",-1,18),new n("á",-1,19),new n("é",-1,20)],f=[new n("id",-1,10),new n("aid",0,9),new n("jaid",1,6),new n("eid",0,9),new n("jeid",3,6),new n("áid",0,7),new n("éid",0,8),new n("i",-1,15),new n("ai",7,14),new n("jai",8,11),new n("ei",7,14),new n("jei",10,11),new n("ái",7,12),new n("éi",7,13),new n("itek",-1,24),new n("eitek",14,21),new n("jeitek",15,20),new n("éitek",14,23),new n("ik",-1,29),new n("aik",18,26),new n("jaik",19,25),new n("eik",18,26),new n("jeik",21,25),new n("áik",18,27),new n("éik",18,28),new n("ink",-1,20),new n("aink",25,17),new n("jaink",26,16),new n("eink",25,17),new n("jeink",28,16),new n("áink",25,18),new n("éink",25,19),new n("aitok",-1,21),new n("jaitok",32,20),new n("áitok",-1,22),new n("im",-1,5),new n("aim",35,4),new n("jaim",36,1),new n("eim",35,4),new n("jeim",38,1),new n("áim",35,2),new n("éim",35,3)],b=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,1,17,52,14],d=new r;function g(){return e<=d.cursor}function h(){var e=d.limit-d.cursor;return!!d.find_among_b(t,23)&&(d.cursor=d.limit-e,!0)}function p(){if(d.cursor>d.limit_backward){d.cursor--,d.ket=d.cursor;var e=d.cursor-1;d.limit_backward<=e&&e<=d.limit&&(d.cursor=e,d.bra=e,d.slice_del())}}function _(){d.ket=d.cursor,d.find_among_b(c,44)&&(d.bra=d.cursor,g()&&(d.slice_del(),function(){var e;if(d.ket=d.cursor,(e=d.find_among_b(a,2))&&(d.bra=d.cursor,g()))switch(e){case 1:d.slice_from(Line truncated
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,i,n;e.it=function(){this.pipeline.reset(),this.pipeline.add(e.it.trimmer,e.it.stopWordFilter,e.it.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.it.stemmer))},e.it.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.it.trimmer=e.trimmerSupport.generateTrimmer(e.it.wordCharacters),e.Pipeline.registerFunction(e.it.trimmer,"trimmer-it"),e.it.stemmer=(r=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=new function(){var e,n,o,t=[new r("",-1,7),new r("qu",0,6),new r("á",0,1),new r("é",0,2),new r("í",0,3),new r("ó",0,4),new r("ú",0,5)],s=[new r("",-1,3),new r("I",0,1),new r("U",0,2)],a=[new r("la",-1,-1),new r("cela",0,-1),new r("gliela",0,-1),new r("mela",0,-1),new r("tela",0,-1),new r("vela",0,-1),new r("le",-1,-1),new r("cele",6,-1),new r("gliele",6,-1),new r("mele",6,-1),new r("tele",6,-1),new r("vele",6,-1),new r("ne",-1,-1),new r("cene",12,-1),new r("gliene",12,-1),new r("mene",12,-1),new r("sene",12,-1),new r("tene",12,-1),new r("vene",12,-1),new r("ci",-1,-1),new r("li",-1,-1),new r("celi",20,-1),new r("glieli",20,-1),new r("meli",20,-1),new r("teli",20,-1),new r("veli",20,-1),new r("gli",20,-1),new r("mi",-1,-1),new r("si",-1,-1),new r("ti",-1,-1),new r("vi",-1,-1),new r("lo",-1,-1),new r("celo",31,-1),new r("glielo",31,-1),new r("melo",31,-1),new r("telo",31,-1),new r("velo",31,-1)],u=[new r("ando",-1,1),new r("endo",-1,1),new r("ar",-1,2),new r("er",-1,2),new r("ir",-1,2)],c=[new r("ic",-1,-1),new r("abil",-1,-1),new r("os",-1,-1),new r("iv",-1,1)],w=[new r("ic",-1,1),new r("abil",-1,1),new r("iv",-1,1)],l=[new r("ica",-1,1),new r("logia",-1,3),new r("osa",-1,1),new r("ista",-1,1),new r("iva",-1,9),new r("anza",-1,1),new r("enza",-1,5),new r("ice",-1,1),new r("atrice",7,1),new r("iche",-1,1),new r("logie",-1,3),new r("abile",-1,1),new r("ibile",-1,1),new r("usione",-1,4),new r("azione",-1,2),new r("uzione",-1,4),new r("atore",-1,2),new r("ose",-1,1),new r("ante",-1,1),new r("mente",-1,1),new r("amente",19,7),new r("iste",-1,1),new r("ive",-1,9),new r("anze",-1,1),new r("enze",-1,5),new r("ici",-1,1),new r("atrici",25,1),new r("ichi",-1,1),new r("abili",-1,1),new r("ibili",-1,1),new r("ismi",-1,1),new r("usioni",-1,4),new r("azioni",-1,2),new r("uzioni",-1,4),new r("atori",-1,2),new r("osi",-1,1),new r("anti",-1,1),new r("amenti",-1,6),new r("imenti",-1,6),new r("isti",-1,1),new r("ivi",-1,9),new r("ico",-1,1),new r("ismo",-1,1),new r("oso",-1,1),new r("amento",-1,6),new r("imento",-1,6),new r("ivo",-1,9),new r("ità",-1,8),new r("istà",-1,1),new r("istè",-1,1),new r("istì",-1,1)],m=[new r("isca",-1,1),new r("enda",-1,1),new r("ata",-1,1),new r("ita",-1,1),new r("uta",-1,1),new r("ava",-1,1),new r("eva",-1,1),new r("iva",-1,1),new r("erebbe",-1,1),new r("irebbe",-1,1),new r("isce",-1,1),new r("ende",-1,1),new r("are",-1,1),new r("ere",-1,1),new r("ire",-1,1),new r("asse",-1,1),new r("ate",-1,1),new r("avate",16,1),new r("evate",16,1),new r("ivate",16,1),new r("ete",-1,1),new r("erete",20,1),new r("irete",20,1),new r("ite",-1,1),new r("ereste",-1,1),new r("ireste",-1,1),new r("ute",-1,1),new r("erai",-1,1),new r("irai",-1,1),new r("isci",-1,1),new r("endi",-1,1),new r("erei",-1,1),new r("irei",-1,1),new r("assi",-1,1),new r("ati",-1,1),new r("iti",-1,1),new r("eresti",-1,1),new r("iresti",-1,1),new r("uti",-1,1),new r("avi",-1,1),new r("evi",-1,1),new r("ivi",-1,1),new r("isco",-1,1),new r("ando",-1,1),new r("endo",-1,1),new r("Yamo",-1,1),new r("iamo",-1,1),new r("avamo",-1,1),new r("evamo",-1,1),new r("ivamo",-1,1),new r("eremo",-1,1),new r("iremo",-1,1),new r("assimo",-1,1),new r("ammo",-1,1),new r("emmo",-1,1),new r("eremmo",54,1),new r("iremmo",54,1),new r("immo",-1,1),new r("ano",-1,1),new r("iscano",58,1),new r("avano",58,1),new r("evano",58,1),new r("ivano",58,1),new r("eranno",-1,1),new r("iranno",-1,1),new r("ono",-1,1),new r("iscono",65,1),new r("arono",65,1),new r("erono",65,1),new r("irono",65,1),new r("erebbero",-1,1),new r("irebbero",-1,1),new r("assero",-1,1),new r("essero",-1,1),new r("issero",-1,1),new r("ato",-1,1),new r("ito",-1,1),new r("uto",-1,1),new r("avo",-1,1),new r("evo",-1,1),new r("ivo",-1,1),new r("ar",-1,1),new r("ir",-1,1),new r("erà",-1,1),new r("irà",-1,1),new r("erò",-1,1),new r("irò",-1,1)],f=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,128,128,8,2,1],v=[17,65,0,0,0,0,0,0,0,0,0,0,0,0,0,128,128,8,2],b=[17],d=new i;function _(e,r,i){return!(!d.eq_s(1,e)||(d.ket=d.cursor,!dLine truncated
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.jp=function(){this.pipeline.reset(),this.pipeline.add(e.jp.stopWordFilter,e.jp.stemmer),r?this.tokenizer=e.jp.tokenizer:(e.tokenizer&&(e.tokenizer=e.jp.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.jp.tokenizer))};var t=new e.TinySegmenter;e.jp.tokenizer=function(n){if(!arguments.length||null==n||null==n)return[];if(Array.isArray(n))return n.map(function(t){return r?new e.Token(t.toLowerCase()):t.toLowerCase()});for(var i=n.toString().toLowerCase().replace(/^\s+/,""),o=i.length-1;o>=0;o--)if(/\S/.test(i.charAt(o))){i=i.substring(0,o+1);break}return t.segment(i).filter(function(e){return!!e}).map(function(t){return r?new e.Token(t):t})},e.jp.stemmer=function(e){return e},e.Pipeline.registerFunction(e.jp.stemmer,"stemmer-jp"),e.jp.wordCharacters="一二三四五六七八九十百千万億兆一-龠々〆ヵヶぁ-んァ-ヴーア-ン゙a-zA-Za-zA-Z0-90-9",e.jp.stopWordFilter=function(t){if(-1===e.jp.stopWordFilter.stopWords.indexOf(r?t.toString():t))return t},e.jp.stopWordFilter=e.generateStopWordFilter("これ それ あれ この その あの ここ そこ あそこ こちら どこ だれ なに なん 何 私 貴方 貴方方 我々 私達 あの人 あのかた 彼女 彼 です あります おります います は が の に を で え から まで より も どの と し それで しかし".split(" ")),e.Pipeline.registerFunction(e.jp.stopWordFilter,"stopWordFilter-jp")}});
@@ -1 +0,0 @@
!function(e,i){"function"==typeof define&&define.amd?define(i):"object"==typeof exports?module.exports=i():i()(e.lunr)}(this,function(){return function(e){e.multiLanguage=function(){for(var i=Array.prototype.slice.call(arguments),t=i.join("-"),r="",n=[],s=[],p=0;p<i.length;++p)"en"==i[p]?(r+="\\w",n.unshift(e.stopWordFilter),n.push(e.stemmer),s.push(e.stemmer)):(r+=e[i[p]].wordCharacters,n.unshift(e[i[p]].stopWordFilter),n.push(e[i[p]].stemmer),s.push(e[i[p]].stemmer));var o=e.trimmerSupport.generateTrimmer(r);return e.Pipeline.registerFunction(o,"lunr-multi-trimmer-"+t),n.unshift(o),function(){this.pipeline.reset(),this.pipeline.add.apply(this.pipeline,n),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add.apply(this.searchPipeline,s))}}}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,n,i;e.no=function(){this.pipeline.reset(),this.pipeline.add(e.no.trimmer,e.no.stopWordFilter,e.no.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.no.stemmer))},e.no.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.no.trimmer=e.trimmerSupport.generateTrimmer(e.no.wordCharacters),e.Pipeline.registerFunction(e.no.trimmer,"trimmer-no"),e.no.stemmer=(r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=new function(){var e,i,t=[new r("a",-1,1),new r("e",-1,1),new r("ede",1,1),new r("ande",1,1),new r("ende",1,1),new r("ane",1,1),new r("ene",1,1),new r("hetene",6,1),new r("erte",1,3),new r("en",-1,1),new r("heten",9,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",12,1),new r("s",-1,2),new r("as",14,1),new r("es",14,1),new r("edes",16,1),new r("endes",16,1),new r("enes",16,1),new r("hetenes",19,1),new r("ens",14,1),new r("hetens",21,1),new r("ers",14,1),new r("ets",14,1),new r("et",-1,1),new r("het",25,1),new r("ert",-1,3),new r("ast",-1,1)],o=[new r("dt",-1,-1),new r("vt",-1,-1)],s=[new r("leg",-1,1),new r("eleg",0,1),new r("ig",-1,1),new r("eig",2,1),new r("lig",2,1),new r("elig",4,1),new r("els",-1,1),new r("lov",-1,1),new r("elov",7,1),new r("slov",7,1),new r("hetslov",9,1)],a=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],m=[119,125,149,1],l=new n;this.setCurrent=function(e){l.setCurrent(e)},this.getCurrent=function(){return l.getCurrent()},this.stem=function(){var r,n,u,d,c=l.cursor;return function(){var r,n=l.cursor+3;if(i=l.limit,0<=n||n<=l.limit){for(e=n;;){if(r=l.cursor,l.in_grouping(a,97,248)){l.cursor=r;break}if(r>=l.limit)return;l.cursor=r+1}for(;!l.out_grouping(a,97,248);){if(l.cursor>=l.limit)return;l.cursor++}(i=l.cursor)<e&&(i=e)}}(),l.limit_backward=c,l.cursor=l.limit,function(){var e,r,n;if(l.cursor>=i&&(r=l.limit_backward,l.limit_backward=i,l.ket=l.cursor,e=l.find_among_b(t,29),l.limit_backward=r,e))switch(l.bra=l.cursor,e){case 1:l.slice_del();break;case 2:n=l.limit-l.cursor,l.in_grouping_b(m,98,122)?l.slice_del():(l.cursor=l.limit-n,l.eq_s_b(1,"k")&&l.out_grouping_b(a,97,248)&&l.slice_del());break;case 3:l.slice_from("er")}}(),l.cursor=l.limit,n=l.limit-l.cursor,l.cursor>=i&&(r=l.limit_backward,l.limit_backward=i,l.ket=l.cursor,l.find_among_b(o,2)?(l.bra=l.cursor,l.limit_backward=r,l.cursor=l.limit-n,l.cursor>l.limit_backward&&(l.cursor--,l.bra=l.cursor,l.slice_del())):l.limit_backward=r),l.cursor=l.limit,l.cursor>=i&&(d=l.limit_backward,l.limit_backward=i,l.ket=l.cursor,(u=l.find_among_b(s,11))?(l.bra=l.cursor,l.limit_backward=d,1==u&&l.slice_del()):l.limit_backward=d),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}),e.Pipeline.registerFunction(e.no.stemmer,"stemmer-no"),e.no.stopWordFilter=e.generateStopWordFilter("alle at av bare begge ble blei bli blir blitt både båe da de deg dei deim deira deires dem den denne der dere deres det dette di din disse ditt du dykk dykkar då eg ein eit eitt eller elles en enn er et ett etter for fordi fra før ha hadde han hans har hennar henne hennes her hjå ho hoe honom hoss hossen hun hva hvem hver hvilke hvilken hvis hvor hvordan hvorfor i ikke ikkje ikkje ingen ingi inkje inn inni ja jeg kan kom korleis korso kun kunne kva kvar kvarhelst kven kvi kvifor man mange me med medan meg meget mellom men mi min mine mitt mot mykje ned no noe noen noka noko nokon nokor nokre nå når og også om opp oss over på samme seg selv si si sia sidan siden sin sine sitt sjøl skal skulle slik so som som somme somt så sånn til um upp ut uten var vart varte ved vere verte vi vil ville vore vors vort vår være være vært å".split(" ")),e.Pipeline.registerFunction(e.no.stopWordFilter,"stopWordFilter-no")}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,s,n;e.pt=function(){this.pipeline.reset(),this.pipeline.add(e.pt.trimmer,e.pt.stopWordFilter,e.pt.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.pt.stemmer))},e.pt.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.pt.trimmer=e.trimmerSupport.generateTrimmer(e.pt.wordCharacters),e.Pipeline.registerFunction(e.pt.trimmer,"trimmer-pt"),e.pt.stemmer=(r=e.stemmerSupport.Among,s=e.stemmerSupport.SnowballProgram,n=new function(){var e,n,i,o=[new r("",-1,3),new r("ã",0,1),new r("õ",0,2)],a=[new r("",-1,3),new r("a~",0,1),new r("o~",0,2)],t=[new r("ic",-1,-1),new r("ad",-1,-1),new r("os",-1,-1),new r("iv",-1,1)],u=[new r("ante",-1,1),new r("avel",-1,1),new r("ível",-1,1)],w=[new r("ic",-1,1),new r("abil",-1,1),new r("iv",-1,1)],m=[new r("ica",-1,1),new r("ância",-1,1),new r("ência",-1,4),new r("ira",-1,9),new r("adora",-1,1),new r("osa",-1,1),new r("ista",-1,1),new r("iva",-1,8),new r("eza",-1,1),new r("logía",-1,2),new r("idade",-1,7),new r("ante",-1,1),new r("mente",-1,6),new r("amente",12,5),new r("ável",-1,1),new r("ível",-1,1),new r("ución",-1,3),new r("ico",-1,1),new r("ismo",-1,1),new r("oso",-1,1),new r("amento",-1,1),new r("imento",-1,1),new r("ivo",-1,8),new r("aça~o",-1,1),new r("ador",-1,1),new r("icas",-1,1),new r("ências",-1,4),new r("iras",-1,9),new r("adoras",-1,1),new r("osas",-1,1),new r("istas",-1,1),new r("ivas",-1,8),new r("ezas",-1,1),new r("logías",-1,2),new r("idades",-1,7),new r("uciones",-1,3),new r("adores",-1,1),new r("antes",-1,1),new r("aço~es",-1,1),new r("icos",-1,1),new r("ismos",-1,1),new r("osos",-1,1),new r("amentos",-1,1),new r("imentos",-1,1),new r("ivos",-1,8)],c=[new r("ada",-1,1),new r("ida",-1,1),new r("ia",-1,1),new r("aria",2,1),new r("eria",2,1),new r("iria",2,1),new r("ara",-1,1),new r("era",-1,1),new r("ira",-1,1),new r("ava",-1,1),new r("asse",-1,1),new r("esse",-1,1),new r("isse",-1,1),new r("aste",-1,1),new r("este",-1,1),new r("iste",-1,1),new r("ei",-1,1),new r("arei",16,1),new r("erei",16,1),new r("irei",16,1),new r("am",-1,1),new r("iam",20,1),new r("ariam",21,1),new r("eriam",21,1),new r("iriam",21,1),new r("aram",20,1),new r("eram",20,1),new r("iram",20,1),new r("avam",20,1),new r("em",-1,1),new r("arem",29,1),new r("erem",29,1),new r("irem",29,1),new r("assem",29,1),new r("essem",29,1),new r("issem",29,1),new r("ado",-1,1),new r("ido",-1,1),new r("ando",-1,1),new r("endo",-1,1),new r("indo",-1,1),new r("ara~o",-1,1),new r("era~o",-1,1),new r("ira~o",-1,1),new r("ar",-1,1),new r("er",-1,1),new r("ir",-1,1),new r("as",-1,1),new r("adas",47,1),new r("idas",47,1),new r("ias",47,1),new r("arias",50,1),new r("erias",50,1),new r("irias",50,1),new r("aras",47,1),new r("eras",47,1),new r("iras",47,1),new r("avas",47,1),new r("es",-1,1),new r("ardes",58,1),new r("erdes",58,1),new r("irdes",58,1),new r("ares",58,1),new r("eres",58,1),new r("ires",58,1),new r("asses",58,1),new r("esses",58,1),new r("isses",58,1),new r("astes",58,1),new r("estes",58,1),new r("istes",58,1),new r("is",-1,1),new r("ais",71,1),new r("eis",71,1),new r("areis",73,1),new r("ereis",73,1),new r("ireis",73,1),new r("áreis",73,1),new r("éreis",73,1),new r("íreis",73,1),new r("ásseis",73,1),new r("ésseis",73,1),new r("ísseis",73,1),new r("áveis",73,1),new r("íeis",73,1),new r("aríeis",84,1),new r("eríeis",84,1),new r("iríeis",84,1),new r("ados",-1,1),new r("idos",-1,1),new r("amos",-1,1),new r("áramos",90,1),new r("éramos",90,1),new r("íramos",90,1),new r("ávamos",90,1),new r("íamos",90,1),new r("aríamos",95,1),new r("eríamos",95,1),new r("iríamos",95,1),new r("emos",-1,1),new r("aremos",99,1),new r("eremos",99,1),new r("iremos",99,1),new r("ássemos",99,1),new r("êssemos",99,1),new r("íssemos",99,1),new r("imos",-1,1),new r("armos",-1,1),new r("ermos",-1,1),new r("irmos",-1,1),new r("ámos",-1,1),new r("arás",-1,1),new r("erás",-1,1),new r("irás",-1,1),new r("eu",-1,1),new r("iu",-1,1),new r("ou",-1,1),new r("ará",-1,1),new r("erá",-1,1),new r("irá",-1,1)],l=[new r("a",-1,1),new r("i",-1,1),new r("o",-1,1),new r("os",-1,1),new r("á",-1,1),new r("í",-1,1),new r("ó",-1,1)],f=[new r("e",-1,1),new r("ç",-1,2),new r("é",-1,1),new r("ê",-1,1)],d=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,3,19,12,2],v=new s;function p(){if(v.out_grouping(d,97,250)){for(;!v.in_grouping(d,97,250);){if(v.cursor>=v.limit)return!0;v.cursor++}return!1}return!0}function _(){var e,r,s=v.cursor;if(v.in_groupLine truncated
-1
View File
@@ -1 +0,0 @@
!function(e,i){"function"==typeof define&&define.amd?define(i):"object"==typeof exports?module.exports=i():i()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var i,r,n;e.ro=function(){this.pipeline.reset(),this.pipeline.add(e.ro.trimmer,e.ro.stopWordFilter,e.ro.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.ro.stemmer))},e.ro.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.ro.trimmer=e.trimmerSupport.generateTrimmer(e.ro.wordCharacters),e.Pipeline.registerFunction(e.ro.trimmer,"trimmer-ro"),e.ro.stemmer=(i=e.stemmerSupport.Among,r=e.stemmerSupport.SnowballProgram,n=new function(){var e,n,t,a,o=[new i("",-1,3),new i("I",0,1),new i("U",0,2)],s=[new i("ea",-1,3),new i("aţia",-1,7),new i("aua",-1,2),new i("iua",-1,4),new i("aţie",-1,7),new i("ele",-1,3),new i("ile",-1,5),new i("iile",6,4),new i("iei",-1,4),new i("atei",-1,6),new i("ii",-1,4),new i("ului",-1,1),new i("ul",-1,1),new i("elor",-1,3),new i("ilor",-1,4),new i("iilor",14,4)],c=[new i("icala",-1,4),new i("iciva",-1,4),new i("ativa",-1,5),new i("itiva",-1,6),new i("icale",-1,4),new i("aţiune",-1,5),new i("iţiune",-1,6),new i("atoare",-1,5),new i("itoare",-1,6),new i("ătoare",-1,5),new i("icitate",-1,4),new i("abilitate",-1,1),new i("ibilitate",-1,2),new i("ivitate",-1,3),new i("icive",-1,4),new i("ative",-1,5),new i("itive",-1,6),new i("icali",-1,4),new i("atori",-1,5),new i("icatori",18,4),new i("itori",-1,6),new i("ători",-1,5),new i("icitati",-1,4),new i("abilitati",-1,1),new i("ivitati",-1,3),new i("icivi",-1,4),new i("ativi",-1,5),new i("itivi",-1,6),new i("icităi",-1,4),new i("abilităi",-1,1),new i("ivităi",-1,3),new i("icităţi",-1,4),new i("abilităţi",-1,1),new i("ivităţi",-1,3),new i("ical",-1,4),new i("ator",-1,5),new i("icator",35,4),new i("itor",-1,6),new i("ător",-1,5),new i("iciv",-1,4),new i("ativ",-1,5),new i("itiv",-1,6),new i("icală",-1,4),new i("icivă",-1,4),new i("ativă",-1,5),new i("itivă",-1,6)],u=[new i("ica",-1,1),new i("abila",-1,1),new i("ibila",-1,1),new i("oasa",-1,1),new i("ata",-1,1),new i("ita",-1,1),new i("anta",-1,1),new i("ista",-1,3),new i("uta",-1,1),new i("iva",-1,1),new i("ic",-1,1),new i("ice",-1,1),new i("abile",-1,1),new i("ibile",-1,1),new i("isme",-1,3),new i("iune",-1,2),new i("oase",-1,1),new i("ate",-1,1),new i("itate",17,1),new i("ite",-1,1),new i("ante",-1,1),new i("iste",-1,3),new i("ute",-1,1),new i("ive",-1,1),new i("ici",-1,1),new i("abili",-1,1),new i("ibili",-1,1),new i("iuni",-1,2),new i("atori",-1,1),new i("osi",-1,1),new i("ati",-1,1),new i("itati",30,1),new i("iti",-1,1),new i("anti",-1,1),new i("isti",-1,3),new i("uti",-1,1),new i("işti",-1,3),new i("ivi",-1,1),new i("ităi",-1,1),new i("oşi",-1,1),new i("ităţi",-1,1),new i("abil",-1,1),new i("ibil",-1,1),new i("ism",-1,3),new i("ator",-1,1),new i("os",-1,1),new i("at",-1,1),new i("it",-1,1),new i("ant",-1,1),new i("ist",-1,3),new i("ut",-1,1),new i("iv",-1,1),new i("ică",-1,1),new i("abilă",-1,1),new i("ibilă",-1,1),new i("oasă",-1,1),new i("ată",-1,1),new i("ită",-1,1),new i("antă",-1,1),new i("istă",-1,3),new i("ută",-1,1),new i("ivă",-1,1)],w=[new i("ea",-1,1),new i("ia",-1,1),new i("esc",-1,1),new i("ăsc",-1,1),new i("ind",-1,1),new i("ând",-1,1),new i("are",-1,1),new i("ere",-1,1),new i("ire",-1,1),new i("âre",-1,1),new i("se",-1,2),new i("ase",10,1),new i("sese",10,2),new i("ise",10,1),new i("use",10,1),new i("âse",10,1),new i("eşte",-1,1),new i("ăşte",-1,1),new i("eze",-1,1),new i("ai",-1,1),new i("eai",19,1),new i("iai",19,1),new i("sei",-1,2),new i("eşti",-1,1),new i("ăşti",-1,1),new i("ui",-1,1),new i("ezi",-1,1),new i("âi",-1,1),new i("aşi",-1,1),new i("seşi",-1,2),new i("aseşi",29,1),new i("seseşi",29,2),new i("iseşi",29,1),new i("useşi",29,1),new i("âseşi",29,1),new i("işi",-1,1),new i("uşi",-1,1),new i("âşi",-1,1),new i("aţi",-1,2),new i("eaţi",38,1),new i("iaţi",38,1),new i("eţi",-1,2),new i("iţi",-1,2),new i("âţi",-1,2),new i("arăţi",-1,1),new i("serăţi",-1,2),new i("aserăţi",45,1),new i("seserăţi",45,2),new i("iserăţi",45,1),new i("userăţi",45,1),new i("âserăţi",45,1),new i("irăţi",-1,1),new i("urăţi",-1,1),new i("ârăţi",-1,1),new i("am",-1,1),new i("eam",54,1),new i("iam",54,1),new i("em",-1,2),new i("asem",57,1),new i("sesem",57,2),new i("isem",57,1),new i("usem",57,1),new i("âsem",57,1),new i("im",-1,2),new i("âm",-1,2),new i("ăm",-1,2),new i("arăm",65,1),new i("serăm",65,2),new i("aserăm",67,1),new i("seserăm",67,2),new i("iserăm",67,1),new i("userăm",67,1),new i("âserăm",67,1),new i("irăm",65,1),newLine truncated
-1
View File
@@ -1 +0,0 @@
!function(e,n){"function"==typeof define&&define.amd?define(n):"object"==typeof exports?module.exports=n():n()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var n,r,t;e.ru=function(){this.pipeline.reset(),this.pipeline.add(e.ru.trimmer,e.ru.stopWordFilter,e.ru.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.ru.stemmer))},e.ru.wordCharacters="Ѐ-҄҇-ԯᴫᵸⷠ-ⷿꙀ-ꚟ︮︯",e.ru.trimmer=e.trimmerSupport.generateTrimmer(e.ru.wordCharacters),e.Pipeline.registerFunction(e.ru.trimmer,"trimmer-ru"),e.ru.stemmer=(n=e.stemmerSupport.Among,r=e.stemmerSupport.SnowballProgram,t=new function(){var e,t,w=[new n("в",-1,1),new n("ив",0,2),new n("ыв",0,2),new n("вши",-1,1),new n("ивши",3,2),new n("ывши",3,2),new n("вшись",-1,1),new n("ившись",6,2),new n("ывшись",6,2)],i=[new n("ее",-1,1),new n("ие",-1,1),new n("ое",-1,1),new n("ые",-1,1),new n("ими",-1,1),new n("ыми",-1,1),new n("ей",-1,1),new n("ий",-1,1),new n("ой",-1,1),new n("ый",-1,1),new n("ем",-1,1),new n("им",-1,1),new n("ом",-1,1),new n("ым",-1,1),new n("его",-1,1),new n("ого",-1,1),new n("ему",-1,1),new n("ому",-1,1),new n("их",-1,1),new n("ых",-1,1),new n("ею",-1,1),new n("ою",-1,1),new n("ую",-1,1),new n("юю",-1,1),new n("ая",-1,1),new n("яя",-1,1)],u=[new n("ем",-1,1),new n("нн",-1,1),new n("вш",-1,1),new n("ивш",2,2),new n("ывш",2,2),new n("щ",-1,1),new n("ющ",5,1),new n("ующ",6,2)],s=[new n("сь",-1,1),new n("ся",-1,1)],o=[new n("ла",-1,1),new n("ила",0,2),new n("ыла",0,2),new n("на",-1,1),new n("ена",3,2),new n("ете",-1,1),new n("ите",-1,2),new n("йте",-1,1),new n("ейте",7,2),new n("уйте",7,2),new n("ли",-1,1),new n("или",10,2),new n("ыли",10,2),new n("й",-1,1),new n("ей",13,2),new n("уй",13,2),new n("л",-1,1),new n("ил",16,2),new n("ыл",16,2),new n("ем",-1,1),new n("им",-1,2),new n("ым",-1,2),new n("н",-1,1),new n("ен",22,2),new n("ло",-1,1),new n("ило",24,2),new n("ыло",24,2),new n("но",-1,1),new n("ено",27,2),new n("нно",27,1),new n("ет",-1,1),new n("ует",30,2),new n("ит",-1,2),new n("ыт",-1,2),new n("ют",-1,1),new n("уют",34,2),new n("ят",-1,2),new n("ны",-1,1),new n("ены",37,2),new n("ть",-1,1),new n("ить",39,2),new n("ыть",39,2),new n("ешь",-1,1),new n("ишь",-1,2),new n("ю",-1,2),new n("ую",44,2)],c=[new n("а",-1,1),new n("ев",-1,1),new n("ов",-1,1),new n("е",-1,1),new n("ие",3,1),new n("ье",3,1),new n("и",-1,1),new n("еи",6,1),new n("ии",6,1),new n("ами",6,1),new n("ями",6,1),new n("иями",10,1),new n("й",-1,1),new n("ей",12,1),new n("ией",13,1),new n("ий",12,1),new n("ой",12,1),new n("ам",-1,1),new n("ем",-1,1),new n("ием",18,1),new n("ом",-1,1),new n("ям",-1,1),new n("иям",21,1),new n("о",-1,1),new n("у",-1,1),new n("ах",-1,1),new n("ях",-1,1),new n("иях",26,1),new n("ы",-1,1),new n("ь",-1,1),new n("ю",-1,1),new n("ию",30,1),new n("ью",30,1),new n("я",-1,1),new n("ия",33,1),new n("ья",33,1)],m=[new n("ост",-1,1),new n("ость",-1,1)],l=[new n("ейше",-1,1),new n("н",-1,2),new n("ейш",-1,1),new n("ь",-1,3)],f=[33,65,8,232],a=new r;function p(){for(;!a.in_grouping(f,1072,1103);){if(a.cursor>=a.limit)return!1;a.cursor++}return!0}function d(){for(;!a.out_grouping(f,1072,1103);){if(a.cursor>=a.limit)return!1;a.cursor++}return!0}function _(e,n){var r,t;if(a.ket=a.cursor,r=a.find_among_b(e,n)){switch(a.bra=a.cursor,r){case 1:if(t=a.limit-a.cursor,!a.eq_s_b(1,"а")&&(a.cursor=a.limit-t,!a.eq_s_b(1,"я")))return!1;case 2:a.slice_del()}return!0}return!1}function b(e,n){var r;return a.ket=a.cursor,!!(r=a.find_among_b(e,n))&&(a.bra=a.cursor,1==r&&a.slice_del(),!0)}function h(){return!!b(i,26)&&(_(u,8),!0)}function g(){var n;a.ket=a.cursor,(n=a.find_among_b(m,2))&&(a.bra=a.cursor,e<=a.cursor&&1==n&&a.slice_del())}this.setCurrent=function(e){a.setCurrent(e)},this.getCurrent=function(){return a.getCurrent()},this.stem=function(){return t=a.limit,e=t,p()&&(t=a.cursor,d()&&p()&&d()&&(e=a.cursor)),a.cursor=a.limit,!(a.cursor<t)&&(a.limit_backward=t,_(w,9)||(a.cursor=a.limit,b(s,2)||(a.cursor=a.limit),h()||(a.cursor=a.limit,_(o,46)||(a.cursor=a.limit,b(c,36)))),a.cursor=a.limit,a.ket=a.cursor,a.eq_s_b(1,"и")?(a.bra=a.cursor,a.slice_del()):a.cursor=a.limit,g(),a.cursor=a.limit,function(){var e;if(a.ket=a.cursor,e=a.find_among_b(l,4))switch(a.bra=a.cursor,e){case 1:if(a.slice_del(),a.ket=a.cursor,!a.eq_s_b(1,"н"))break;a.bra=a.cursor;case 2:if(!a.eq_s_b(1,"н"))break;case 3:a.slice_del()}}(),!0)}},function(e){return"function"==typeof e.update?e.update(function(e){return t.setCLine truncated
@@ -1 +0,0 @@
!function(r,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(r.lunr)}(this,function(){return function(r){r.stemmerSupport={Among:function(r,t,i,s){if(this.toCharArray=function(r){for(var t=r.length,i=new Array(t),s=0;s<t;s++)i[s]=r.charCodeAt(s);return i},!r&&""!=r||!t&&0!=t||!i)throw"Bad Among initialisation: s:"+r+", substring_i: "+t+", result: "+i;this.s_size=r.length,this.s=this.toCharArray(r),this.substring_i=t,this.result=i,this.method=s},SnowballProgram:function(){var r;return{bra:0,ket:0,limit:0,cursor:0,limit_backward:0,setCurrent:function(t){r=t,this.cursor=0,this.limit=t.length,this.limit_backward=0,this.bra=this.cursor,this.ket=this.limit},getCurrent:function(){var t=r;return r=null,t},in_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e<=s&&e>=i&&t[(e-=i)>>3]&1<<(7&e))return this.cursor++,!0}return!1},in_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e<=s&&e>=i&&t[(e-=i)>>3]&1<<(7&e))return this.cursor--,!0}return!1},out_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e>s||e<i)return this.cursor++,!0;if(!(t[(e-=i)>>3]&1<<(7&e)))return this.cursor++,!0}return!1},out_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e>s||e<i)return this.cursor--,!0;if(!(t[(e-=i)>>3]&1<<(7&e)))return this.cursor--,!0}return!1},eq_s:function(t,i){if(this.limit-this.cursor<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor+s)!=i.charCodeAt(s))return!1;return this.cursor+=t,!0},eq_s_b:function(t,i){if(this.cursor-this.limit_backward<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor-t+s)!=i.charCodeAt(s))return!1;return this.cursor-=t,!0},find_among:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=l;m<_.s_size;m++){if(n+l==u){f=-1;break}if(f=r.charCodeAt(n+l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){if(o>=(_=t[s]).s_size){if(this.cursor=n+_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n+_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},find_among_b:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit_backward,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=(m=t[a]).s_size-1-l;_>=0;_--){if(n-l==u){f=-1;break}if(f=r.charCodeAt(n-1-l)-m.s[_])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var m;if(o>=(m=t[s]).s_size){if(this.cursor=n-m.s_size,!m.method)return m.result;var b=m.method();if(this.cursor=n-m.s_size,b)return m.result}if((s=m.substring_i)<0)return 0}},replace_s:function(t,i,s){var e=s.length-(i-t),n=r.substring(0,t),u=r.substring(i);return r=n+s+u,this.limit+=e,this.cursor>=i?this.cursor+=e:this.cursor>t&&(this.cursor=t),e},slice_check:function(){if(this.bra<0||this.bra>this.ket||this.ket>this.limit||this.limit>r.length)throw"faulty slice operation"},slice_from:function(r){this.slice_check(),this.replace_s(this.bra,this.ket,r)},slice_del:function(){this.slice_from("")},insert:function(r,t,i){var s=this.replace_s(r,t,i);r<=this.bra&&(this.bra+=s),r<=this.ket&&(this.ket+=s)},slice_to:function(){return this.slice_check(),r.substring(this.bra,this.ket)},eq_v_b:function(r){return this.eq_s_b(r.length,r)}}}},r.trimmerSupport={generateTrimmer:function(r){var t=new RegExp("^[^"+r+"]+"),i=new RegExp("[^"+r+"]+$");return function(r){return"function"==typeof r.update?r.update(function(r){return r.replace(t,"").replace(i,"")}):r.replace(t,"").replace(i,"")}}}}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,n,t;e.sv=function(){this.pipeline.reset(),this.pipeline.add(e.sv.trimmer,e.sv.stopWordFilter,e.sv.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sv.stemmer))},e.sv.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.sv.trimmer=e.trimmerSupport.generateTrimmer(e.sv.wordCharacters),e.Pipeline.registerFunction(e.sv.trimmer,"trimmer-sv"),e.sv.stemmer=(r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,t=new function(){var e,t,i=[new r("a",-1,1),new r("arna",0,1),new r("erna",0,1),new r("heterna",2,1),new r("orna",0,1),new r("ad",-1,1),new r("e",-1,1),new r("ade",6,1),new r("ande",6,1),new r("arne",6,1),new r("are",6,1),new r("aste",6,1),new r("en",-1,1),new r("anden",12,1),new r("aren",12,1),new r("heten",12,1),new r("ern",-1,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",18,1),new r("or",-1,1),new r("s",-1,2),new r("as",21,1),new r("arnas",22,1),new r("ernas",22,1),new r("ornas",22,1),new r("es",21,1),new r("ades",26,1),new r("andes",26,1),new r("ens",21,1),new r("arens",29,1),new r("hetens",29,1),new r("erns",21,1),new r("at",-1,1),new r("andet",-1,1),new r("het",-1,1),new r("ast",-1,1)],s=[new r("dd",-1,-1),new r("gd",-1,-1),new r("nn",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1),new r("tt",-1,-1)],a=[new r("ig",-1,1),new r("lig",0,1),new r("els",-1,1),new r("fullt",-1,3),new r("löst",-1,2)],o=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,24,0,32],u=[119,127,149],m=new n;this.setCurrent=function(e){m.setCurrent(e)},this.getCurrent=function(){return m.getCurrent()},this.stem=function(){var r,n=m.cursor;return function(){var r,n=m.cursor+3;if(t=m.limit,0<=n||n<=m.limit){for(e=n;;){if(r=m.cursor,m.in_grouping(o,97,246)){m.cursor=r;break}if(m.cursor=r,m.cursor>=m.limit)return;m.cursor++}for(;!m.out_grouping(o,97,246);){if(m.cursor>=m.limit)return;m.cursor++}(t=m.cursor)<e&&(t=e)}}(),m.limit_backward=n,m.cursor=m.limit,function(){var e,r=m.limit_backward;if(m.cursor>=t&&(m.limit_backward=t,m.cursor=m.limit,m.ket=m.cursor,e=m.find_among_b(i,37),m.limit_backward=r,e))switch(m.bra=m.cursor,e){case 1:m.slice_del();break;case 2:m.in_grouping_b(u,98,121)&&m.slice_del()}}(),m.cursor=m.limit,r=m.limit_backward,m.cursor>=t&&(m.limit_backward=t,m.cursor=m.limit,m.find_among_b(s,7)&&(m.cursor=m.limit,m.ket=m.cursor,m.cursor>m.limit_backward&&(m.bra=--m.cursor,m.slice_del())),m.limit_backward=r),m.cursor=m.limit,function(){var e,r;if(m.cursor>=t){if(r=m.limit_backward,m.limit_backward=t,m.cursor=m.limit,m.ket=m.cursor,e=m.find_among_b(a,5))switch(m.bra=m.cursor,e){case 1:m.slice_del();break;case 2:m.slice_from("lös");break;case 3:m.slice_from("full")}m.limit_backward=r}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return t.setCurrent(e),t.stem(),t.getCurrent()}):(t.setCurrent(e),t.stem(),t.getCurrent())}),e.Pipeline.registerFunction(e.sv.stemmer,"stemmer-sv"),e.sv.stopWordFilter=e.generateStopWordFilter("alla allt att av blev bli blir blivit de dem den denna deras dess dessa det detta dig din dina ditt du där då efter ej eller en er era ert ett från för ha hade han hans har henne hennes hon honom hur här i icke ingen inom inte jag ju kan kunde man med mellan men mig min mina mitt mot mycket ni nu när någon något några och om oss på samma sedan sig sin sina sitta själv skulle som så sådan sådana sådant till under upp ut utan vad var vara varför varit varje vars vart vem vi vid vilka vilkas vilken vilket vår våra vårt än är åt över".split(" ")),e.Pipeline.registerFunction(e.sv.stopWordFilter,"stopWordFilter-sv")}});
-1
View File
@@ -1 +0,0 @@
!function(r,i){"function"==typeof define&&define.amd?define(i):"object"==typeof exports?module.exports=i():i()(r.lunr)}(this,function(){return function(r){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var i,e,n;r.tr=function(){this.pipeline.reset(),this.pipeline.add(r.tr.trimmer,r.tr.stopWordFilter,r.tr.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(r.tr.stemmer))},r.tr.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",r.tr.trimmer=r.trimmerSupport.generateTrimmer(r.tr.wordCharacters),r.Pipeline.registerFunction(r.tr.trimmer,"trimmer-tr"),r.tr.stemmer=(i=r.stemmerSupport.Among,e=r.stemmerSupport.SnowballProgram,n=new function(){var r,n=[new i("m",-1,-1),new i("n",-1,-1),new i("miz",-1,-1),new i("niz",-1,-1),new i("muz",-1,-1),new i("nuz",-1,-1),new i("müz",-1,-1),new i("nüz",-1,-1),new i("mız",-1,-1),new i("nız",-1,-1)],t=[new i("leri",-1,-1),new i("ları",-1,-1)],u=[new i("ni",-1,-1),new i("nu",-1,-1),new i("nü",-1,-1),new i("nı",-1,-1)],o=[new i("in",-1,-1),new i("un",-1,-1),new i("ün",-1,-1),new i("ın",-1,-1)],s=[new i("a",-1,-1),new i("e",-1,-1)],c=[new i("na",-1,-1),new i("ne",-1,-1)],l=[new i("da",-1,-1),new i("ta",-1,-1),new i("de",-1,-1),new i("te",-1,-1)],a=[new i("nda",-1,-1),new i("nde",-1,-1)],m=[new i("dan",-1,-1),new i("tan",-1,-1),new i("den",-1,-1),new i("ten",-1,-1)],d=[new i("ndan",-1,-1),new i("nden",-1,-1)],f=[new i("la",-1,-1),new i("le",-1,-1)],b=[new i("ca",-1,-1),new i("ce",-1,-1)],w=[new i("im",-1,-1),new i("um",-1,-1),new i("üm",-1,-1),new i("ım",-1,-1)],_=[new i("sin",-1,-1),new i("sun",-1,-1),new i("sün",-1,-1),new i("sın",-1,-1)],k=[new i("iz",-1,-1),new i("uz",-1,-1),new i("üz",-1,-1),new i("ız",-1,-1)],p=[new i("siniz",-1,-1),new i("sunuz",-1,-1),new i("sünüz",-1,-1),new i("sınız",-1,-1)],g=[new i("lar",-1,-1),new i("ler",-1,-1)],y=[new i("niz",-1,-1),new i("nuz",-1,-1),new i("nüz",-1,-1),new i("nız",-1,-1)],z=[new i("dir",-1,-1),new i("tir",-1,-1),new i("dur",-1,-1),new i("tur",-1,-1),new i("dür",-1,-1),new i("tür",-1,-1),new i("dır",-1,-1),new i("tır",-1,-1)],h=[new i("casına",-1,-1),new i("cesine",-1,-1)],v=[new i("di",-1,-1),new i("ti",-1,-1),new i("dik",-1,-1),new i("tik",-1,-1),new i("duk",-1,-1),new i("tuk",-1,-1),new i("dük",-1,-1),new i("tük",-1,-1),new i("dık",-1,-1),new i("tık",-1,-1),new i("dim",-1,-1),new i("tim",-1,-1),new i("dum",-1,-1),new i("tum",-1,-1),new i("düm",-1,-1),new i("tüm",-1,-1),new i("dım",-1,-1),new i("tım",-1,-1),new i("din",-1,-1),new i("tin",-1,-1),new i("dun",-1,-1),new i("tun",-1,-1),new i("dün",-1,-1),new i("tün",-1,-1),new i("dın",-1,-1),new i("tın",-1,-1),new i("du",-1,-1),new i("tu",-1,-1),new i("dü",-1,-1),new i("tü",-1,-1),new i("dı",-1,-1),new i("tı",-1,-1)],q=[new i("sa",-1,-1),new i("se",-1,-1),new i("sak",-1,-1),new i("sek",-1,-1),new i("sam",-1,-1),new i("sem",-1,-1),new i("san",-1,-1),new i("sen",-1,-1)],C=[new i("miş",-1,-1),new i("muş",-1,-1),new i("müş",-1,-1),new i("mış",-1,-1)],P=[new i("b",-1,1),new i("c",-1,2),new i("d",-1,3),new i("ğ",-1,4)],F=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,32,8,0,0,0,0,0,0,1],S=[1,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,1],W=[65],L=[65],x=[["a",[1,64,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],97,305],["e",[17,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,130],101,252],["ı",[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],97,305],["i",[17],101,105],["o",W,111,117],["ö",L,246,252],["u",W,111,117]],A=new e;function E(r,i,e){for(;;){var n=A.limit-A.cursor;if(A.in_grouping_b(r,i,e)){A.cursor=A.limit-n;break}if(A.cursor=A.limit-n,A.cursor<=A.limit_backward)return!1;A.cursor--}return!0}function j(){var r,i;r=A.limit-A.cursor,E(F,97,305);for(var e=0;e<x.length;e++){i=A.limit-A.cursor;var n=x[e];if(A.eq_s_b(1,n[0])&&E(n[1],n[2],n[3]))return A.cursor=A.limit-r,!0;A.cursor=A.limit-i}return A.cursor=A.limit-i,!(!A.eq_s_b(1,"ü")||!E(L,246,252))&&(A.cursor=A.limit-r,!0)}function T(r,i){var e,n=A.limit-A.cursor;return r()&&(A.cursor=A.limit-n,A.cursor>A.limit_backward&&(A.cursor--,e=A.limit-A.cursor,i()))?(A.cursor=A.limit-e,!0):(A.cursor=A.limit-n,r()?(A.cursor=A.limit-n,!1):(A.cursor=A.limit-n,!(A.cursor<=A.limit_backward)&&(A.cursor--,!!i()&&(A.cursor=A.limit-n,!0))))}function Z(r){return T(r,function(){return A.in_grouping_b(F,97,305)})}function B(){return Z(function(){return A.eq_s_b(1,"n")})}function D(){return Z(function(){return A.eq_s_b(1,"y")})}function G(){return A.find_among_b(n,10)&&T(function(){return A.in_grouping_b(S,105,305)},function(){return A.out_grouping_b(F,97,305)})}function H(){return j()&&A.in_grouping_b(S,105,305)&&Z(function(){return A.eq_sLine truncated
-1
View File
@@ -1 +0,0 @@
!function(_,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(_.lunr)}(this,function(){return function(_){function t(){var _={"[一二三四五六七八九十百千万億兆]":"M","[一-龠々〆ヵヶ]":"H","[ぁ-ん]":"I","[ァ-ヴーア-ン゙ー]":"K","[a-zA-Za-zA-Z]":"A","[0-90-9]":"N"};for(var t in this.chartype_=[],_){var H=new RegExp;H.compile(t),this.chartype_.push([H,_[t]])}return this.BIAS__=-332,this.BC1__={HH:6,II:2461,KH:406,OH:-1378},this.BC2__={AA:-3267,AI:2744,AN:-878,HH:-4070,HM:-1711,HN:4012,HO:3761,IA:1327,IH:-1184,II:-1332,IK:1721,IO:5492,KI:3831,KK:-8741,MH:-3132,MK:3334,OO:-2920},this.BC3__={HH:996,HI:626,HK:-721,HN:-1307,HO:-836,IH:-301,KK:2762,MK:1079,MM:4034,OA:-1652,OH:266},this.BP1__={BB:295,OB:304,OO:-125,UB:352},this.BP2__={BO:60,OO:-1762},this.BQ1__={BHH:1150,BHM:1521,BII:-1158,BIM:886,BMH:1208,BNH:449,BOH:-91,BOO:-2597,OHI:451,OIH:-296,OKA:1851,OKH:-1020,OKK:904,OOO:2965},this.BQ2__={BHH:118,BHI:-1159,BHM:466,BIH:-919,BKK:-1720,BKO:864,OHH:-1139,OHM:-181,OIH:153,UHI:-1146},this.BQ3__={BHH:-792,BHI:2664,BII:-299,BKI:419,BMH:937,BMM:8335,BNN:998,BOH:775,OHH:2174,OHM:439,OII:280,OKH:1798,OKI:-793,OKO:-2242,OMH:-2402,OOO:11699},this.BQ4__={BHH:-3895,BIH:3761,BII:-4654,BIK:1348,BKK:-1806,BMI:-3385,BOO:-12396,OAH:926,OHH:266,OHK:-2036,ONN:-973},this.BW1__={",と":660,",同":727,"B1あ":1404,"B1同":542,"、と":660,"、同":727,"」と":1682,"あっ":1505,"いう":1743,"いっ":-2055,"いる":672,"うし":-4817,"うん":665,"から":3472,"がら":600,"こう":-790,"こと":2083,"こん":-1262,"さら":-4143,"さん":4573,"した":2641,"して":1104,"すで":-3399,"そこ":1977,"それ":-871,"たち":1122,"ため":601,"った":3463,"つい":-802,"てい":805,"てき":1249,"でき":1127,"です":3445,"では":844,"とい":-4915,"とみ":1922,"どこ":3887,"ない":5713,"なっ":3015,"など":7379,"なん":-1113,"にし":2468,"には":1498,"にも":1671,"に対":-912,"の一":-501,"の中":741,"ませ":2448,"まで":1711,"まま":2600,"まる":-2155,"やむ":-1947,"よっ":-2565,"れた":2369,"れで":-913,"をし":1860,"を見":731,"亡く":-1886,"京都":2558,"取り":-2784,"大き":-2604,"大阪":1497,"平方":-2314,"引き":-1336,"日本":-195,"本当":-2423,"毎日":-2113,"目指":-724,"B1あ":1404,"B1同":542,"」と":1682},this.BW2__={"..":-11822,11:-669,"――":-5730,"−−":-13175,"いう":-1609,"うか":2490,"かし":-1350,"かも":-602,"から":-7194,"かれ":4612,"がい":853,"がら":-3198,"きた":1941,"くな":-1597,"こと":-8392,"この":-4193,"させ":4533,"され":13168,"さん":-3977,"しい":-1819,"しか":-545,"した":5078,"して":972,"しな":939,"その":-3744,"たい":-1253,"たた":-662,"ただ":-3857,"たち":-786,"たと":1224,"たは":-939,"った":4589,"って":1647,"っと":-2094,"てい":6144,"てき":3640,"てく":2551,"ては":-3110,"ても":-3065,"でい":2666,"でき":-1528,"でし":-3828,"です":-4761,"でも":-4203,"とい":1890,"とこ":-1746,"とと":-2279,"との":720,"とみ":5168,"とも":-3941,"ない":-2488,"なが":-1313,"など":-6509,"なの":2614,"なん":3099,"にお":-1615,"にし":2748,"にな":2454,"によ":-7236,"に対":-14943,"に従":-4688,"に関":-11388,"のか":2093,"ので":-7059,"のに":-6041,"のの":-6125,"はい":1073,"はが":-1033,"はず":-2532,"ばれ":1813,"まし":-1316,"まで":-6621,"まれ":5409,"めて":-3153,"もい":2230,"もの":-10713,"らか":-944,"らし":-1611,"らに":-1897,"りし":651,"りま":1620,"れた":4270,"れて":849,"れば":4114,"ろう":6067,"われ":7901,"を通":-11877,"んだ":728,"んな":-4115,"一人":602,"一方":-1375,"一日":970,"一部":-1051,"上が":-4479,"会社":-1116,"出て":2163,"分の":-7758,"同党":970,"同日":-913,"大阪":-2471,"委員":-1250,"少な":-1050,"年度":-8669,"年間":-1626,"府県":-2363,"手権":-1982,"新聞":-4066,"日新":-722,"日本":-7068,"日米":3372,"曜日":-601,"朝鮮":-2355,"本人":-2697,"東京":-1543,"然と":-1384,"社会":-1276,"立て":-990,"第に":-1612,"米国":-4268,"11":-669},this.BW3__={"あた":-2194,"あり":719,"ある":3846,"い.":-1185,"い。":-1185,"いい":5308,"いえ":2079,"いく":3029,"いた":2056,"いっ":1883,"いる":5600,"いわ":1527,"うち":1117,"うと":4798,"えと":1454,"か.":2857,"か。":2857,"かけ":-743,"かっ":-4098,"かに":-669,"から":6520,"かり":-2670,"が,":1816,"が、":1816,"がき":-4855,"がけ":-1127,"がっ":-913,"がら":-4977,"がり":-2064,"きた":1645,"けど":1374,"こと":7397,"この":1542,"ころ":-2757,"さい":-714,"さを":976,"し,":1557,"し、":1557,"しい":-3714,"した":3562,"して":1449,"しな":2608,"しま":1200,"す.":-1310,"す。":-1310,"する":6521,"ず,":3426,"ず、":3426,"ずに":841,"そう":428,"た.":8875,"た。":8875,"たい":-594,"たの":812,"たり":-1183,"たる":-853,"だ.":4098,"だ。":4098,"だっ":1004,"った":-4748,"って":300,"てい":6240,"てお":855,"てLine truncated
@@ -1 +0,0 @@
!function(e,t){for(var n in t)e[n]=t[n]}(window,function(e){function t(r){if(n[r])return n[r].exports;var o=n[r]={i:r,l:!1,exports:{}};return e[r].call(o.exports,o,o.exports,t),o.l=!0,o.exports}var n={};return t.m=e,t.c=n,t.d=function(e,n,r){t.o(e,n)||Object.defineProperty(e,n,{configurable:!1,enumerable:!0,get:r})},t.n=function(e){var n=e&&e.__esModule?function(){return e.default}:function(){return e};return t.d(n,"a",n),n},t.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)},t.p="",t(t.s=4)}({4:function(e,t,n){"use strict";n(5)},5:function(e,t){!function(t){!function(e,t,n){function r(e,t){return typeof e===t}function o(e){var t=_.className,n=C._config.classPrefix||"";if(T&&(t=t.baseVal),C._config.enableJSClass){var r=new RegExp("(^|\\s)"+n+"no-js(\\s|$)");t=t.replace(r,"$1"+n+"js$2")}C._config.enableClasses&&(t+=" "+n+e.join(" "+n),T?_.className.baseVal=t:_.className=t)}function i(e,t){if("object"==typeof e)for(var n in e)b(e,n)&&i(n,e[n]);else{e=e.toLowerCase();var r=e.split("."),s=C[r[0]];if(2==r.length&&(s=s[r[1]]),void 0!==s)return C;t="function"==typeof t?t():t,1==r.length?C[r[0]]=t:(!C[r[0]]||C[r[0]]instanceof Boolean||(C[r[0]]=new Boolean(C[r[0]])),C[r[0]][r[1]]=t),o([(t&&0!=t?"":"no-")+r.join("-")]),C._trigger(e,t)}return C}function s(){return"function"!=typeof t.createElement?t.createElement(arguments[0]):T?t.createElementNS.call(t,"http://www.w3.org/2000/svg",arguments[0]):t.createElement.apply(t,arguments)}function a(){var e=t.body;return e||(e=s(T?"svg":"body"),e.fake=!0),e}function u(e,n,r,o){var i,u,l,f,c="modernizr",d=s("div"),p=a();if(parseInt(r,10))for(;r--;)l=s("div"),l.id=o?o[r]:c+(r+1),d.appendChild(l);return i=s("style"),i.type="text/css",i.id="s"+c,(p.fake?p:d).appendChild(i),p.appendChild(d),i.styleSheet?i.styleSheet.cssText=e:i.appendChild(t.createTextNode(e)),d.id=c,p.fake&&(p.style.background="",p.style.overflow="hidden",f=_.style.overflow,_.style.overflow="hidden",_.appendChild(p)),u=n(d,e),p.fake?(p.parentNode.removeChild(p),_.style.overflow=f,_.offsetHeight):d.parentNode.removeChild(d),!!u}function l(e,t){return!!~(""+e).indexOf(t)}function f(e){return e.replace(/([A-Z])/g,function(e,t){return"-"+t.toLowerCase()}).replace(/^ms-/,"-ms-")}function c(t,n,r){var o;if("getComputedStyle"in e){o=getComputedStyle.call(e,t,n);var i=e.console;if(null!==o)r&&(o=o.getPropertyValue(r));else if(i){var s=i.error?"error":"log";i[s].call(i,"getComputedStyle returning null, its possible modernizr test results are inaccurate")}}else o=!n&&t.currentStyle&&t.currentStyle[r];return o}function d(t,r){var o=t.length;if("CSS"in e&&"supports"in e.CSS){for(;o--;)if(e.CSS.supports(f(t[o]),r))return!0;return!1}if("CSSSupportsRule"in e){for(var i=[];o--;)i.push("("+f(t[o])+":"+r+")");return i=i.join(" or "),u("@supports ("+i+") { #modernizr { position: absolute; } }",function(e){return"absolute"==c(e,null,"position")})}return n}function p(e){return e.replace(/([a-z])-([a-z])/g,function(e,t,n){return t+n.toUpperCase()}).replace(/^-/,"")}function h(e,t,o,i){function a(){f&&(delete j.style,delete j.modElem)}if(i=!r(i,"undefined")&&i,!r(o,"undefined")){var u=d(e,o);if(!r(u,"undefined"))return u}for(var f,c,h,m,v,g=["modernizr","tspan","samp"];!j.style&&g.length;)f=!0,j.modElem=s(g.shift()),j.style=j.modElem.style;for(h=e.length,c=0;h>c;c++)if(m=e[c],v=j.style[m],l(m,"-")&&(m=p(m)),j.style[m]!==n){if(i||r(o,"undefined"))return a(),"pfx"!=t||m;try{j.style[m]=o}catch(e){}if(j.style[m]!=v)return a(),"pfx"!=t||m}return a(),!1}function m(e,t){return function(){return e.apply(t,arguments)}}function v(e,t,n){var o;for(var i in e)if(e[i]in t)return!1===n?e[i]:(o=t[e[i]],r(o,"function")?m(o,n||t):o);return!1}function g(e,t,n,o,i){var s=e.charAt(0).toUpperCase()+e.slice(1),a=(e+" "+k.join(s+" ")+s).split(" ");return r(t,"string")||r(t,"undefined")?h(a,t,o,i):(a=(e+" "+A.join(s+" ")+s).split(" "),v(a,t,n))}function y(e,t,r){return g(e,n,n,t,r)}var w=[],S={_version:"3.5.0",_config:{classPrefix:"",enableClasses:!0,enableJSClass:!0,usePrefixes:!0},_q:[],on:function(e,t){var n=this;setTimeout(function(){t(n[e])},0)},addTest:function(e,t,n){w.push({name:e,fn:t,options:n})},addAsyncTest:function(e){w.push({name:null,fn:e})}},C=function(){};C.prototype=S,C=new C;var b,x=[],_=t.documentElement,T="svg"===_.nodeName.toLowerCase();!function(){var e={}.hasOwnProperty;b=r(e,"undefined")||r(e.call,"undefined")?function(e,t){return t in e&&r(e.constructor.prototype[t],"undefined")}:function(t,n){return e.call(t,n)}}(),S._l={},S.on=function(e,t){this._l[e]||(this._l[e]=[]),this._l[e].push(t),C.hasOwnProperty(e)&&setTimeout(function(){C._trigger(e,C[e])},0)},S._trigger=function(e,t){if(this._l[e]){var n=this._l[e];setTimeout(function(){var e;for(e=0;e<n.length;e++)(0,n[e])(t)},0),delete this._l[e]}},C._q.push(function(){S.addTest=i}),C.addTest("json","JSON"in e&&"parse"in JSON&&"stringify"in JSON),C.addTest("svg",!!t.createElementNS&&!!t.createElementNS("http://www.w3.org/2000/svg","svg").createSVGRect);var P=S.testStylesLine truncated
@@ -1,2 +0,0 @@
button[data-md-color-accent],button[data-md-color-primary]{width:13rem;margin-bottom:.4rem;padding:2.4rem .8rem .4rem;transition:background-color .25s,opacity .25s;border-radius:.2rem;color:#fff;font-size:1.28rem;text-align:left;cursor:pointer}button[data-md-color-accent]:hover,button[data-md-color-primary]:hover{opacity:.75}button[data-md-color-primary=red]{background-color:#ef5350}[data-md-color-primary=red] .md-typeset a{color:#ef5350}[data-md-color-primary=red] .md-header,[data-md-color-primary=red] .md-hero{background-color:#ef5350}[data-md-color-primary=red] .md-nav__link--active,[data-md-color-primary=red] .md-nav__link:active{color:#ef5350}[data-md-color-primary=red] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=pink]{background-color:#e91e63}[data-md-color-primary=pink] .md-typeset a{color:#e91e63}[data-md-color-primary=pink] .md-header,[data-md-color-primary=pink] .md-hero{background-color:#e91e63}[data-md-color-primary=pink] .md-nav__link--active,[data-md-color-primary=pink] .md-nav__link:active{color:#e91e63}[data-md-color-primary=pink] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=purple]{background-color:#ab47bc}[data-md-color-primary=purple] .md-typeset a{color:#ab47bc}[data-md-color-primary=purple] .md-header,[data-md-color-primary=purple] .md-hero{background-color:#ab47bc}[data-md-color-primary=purple] .md-nav__link--active,[data-md-color-primary=purple] .md-nav__link:active{color:#ab47bc}[data-md-color-primary=purple] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=deep-purple]{background-color:#7e57c2}[data-md-color-primary=deep-purple] .md-typeset a{color:#7e57c2}[data-md-color-primary=deep-purple] .md-header,[data-md-color-primary=deep-purple] .md-hero{background-color:#7e57c2}[data-md-color-primary=deep-purple] .md-nav__link--active,[data-md-color-primary=deep-purple] .md-nav__link:active{color:#7e57c2}[data-md-color-primary=deep-purple] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=indigo]{background-color:#3f51b5}[data-md-color-primary=indigo] .md-typeset a{color:#3f51b5}[data-md-color-primary=indigo] .md-header,[data-md-color-primary=indigo] .md-hero{background-color:#3f51b5}[data-md-color-primary=indigo] .md-nav__link--active,[data-md-color-primary=indigo] .md-nav__link:active{color:#3f51b5}[data-md-color-primary=indigo] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=blue]{background-color:#2196f3}[data-md-color-primary=blue] .md-typeset a{color:#2196f3}[data-md-color-primary=blue] .md-header,[data-md-color-primary=blue] .md-hero{background-color:#2196f3}[data-md-color-primary=blue] .md-nav__link--active,[data-md-color-primary=blue] .md-nav__link:active{color:#2196f3}[data-md-color-primary=blue] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=light-blue]{background-color:#03a9f4}[data-md-color-primary=light-blue] .md-typeset a{color:#03a9f4}[data-md-color-primary=light-blue] .md-header,[data-md-color-primary=light-blue] .md-hero{background-color:#03a9f4}[data-md-color-primary=light-blue] .md-nav__link--active,[data-md-color-primary=light-blue] .md-nav__link:active{color:#03a9f4}[data-md-color-primary=light-blue] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=cyan]{background-color:#00bcd4}[data-md-color-primary=cyan] .md-typeset a{color:#00bcd4}[data-md-color-primary=cyan] .md-header,[data-md-color-primary=cyan] .md-hero{background-color:#00bcd4}[data-md-color-primary=cyan] .md-nav__link--active,[data-md-color-primary=cyan] .md-nav__link:active{color:#00bcd4}[data-md-color-primary=cyan] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=teal]{background-color:#009688}[data-md-color-primary=teal] .md-typeset a{color:#009688}[data-md-color-primary=teal] .md-header,[data-md-color-primary=teal] .md-hero{background-color:#009688}[data-md-color-primary=teal] .md-nav__link--active,[data-md-color-primary=teal] .md-nav__link:active{color:#009688}[data-md-color-primary=teal] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=green]{background-color:#4caf50}[data-md-color-primary=green] .md-typeset a{color:#4caf50}[data-md-color-primary=green] .md-header,[data-md-color-primary=green] .md-hero{background-color:#4caf50}[data-md-color-primary=green] .md-nav__link--active,[data-md-color-primary=green] .md-nav__link:active{color:#4caf50}[data-md-color-primary=green] .md-nav__item--nested>.md-nav__link{color:inherit}button[data-md-color-primary=light-green]{background-color:#7cb342}[data-md-color-primary=light-green] .md-typeset a{color:#7cb342}[data-md-color-primary=light-green] .md-header,[data-md-color-primary=light-green] .md-hero{background-color:#7cb342}[data-md-color-primary=light-green] .md-nav__link--active,[data-md-color-primary=light-green] .md-nav__link:active{color:#7cb342}[data-md-color-primary=light-green] Line truncated
/*# sourceMappingURL=data:application/json;charset=utf-8;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbXSwibmFtZXMiOltdLCJtYXBwaW5ncyI6IiIsImZpbGUiOiJhc3NldHMvc3R5bGVzaGVldHMvYXBwbGljYXRpb24tcGFsZXR0ZS42MDc5NDc2Yy5jc3MiLCJzb3VyY2VSb290IjoiIn0=*/
@@ -1,2 +0,0 @@
html{box-sizing:border-box}*,:after,:before{box-sizing:inherit}html{-webkit-text-size-adjust:none;-moz-text-size-adjust:none;-ms-text-size-adjust:none;text-size-adjust:none}body{margin:0}hr{overflow:visible;box-sizing:content-box}a{-webkit-text-decoration-skip:objects}a,button,input,label{-webkit-tap-highlight-color:transparent}a{color:inherit;text-decoration:none}small,sub,sup{font-size:80%}sub,sup{position:relative;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}img{border-style:none}table{border-collapse:separate;border-spacing:0}td,th{font-weight:400;vertical-align:top}button{margin:0;padding:0;border:0;outline-style:none;background:transparent;font-size:inherit}input{border:0;outline:0}.md-clipboard:before,.md-icon,.md-nav__button,.md-nav__link:after,.md-nav__title:before,.md-search-result__article--document:before,.md-source-file:before,.md-typeset .admonition>.admonition-title:before,.md-typeset .admonition>summary:before,.md-typeset .critic.comment:before,.md-typeset .footnote-backref,.md-typeset .task-list-control .task-list-indicator:before,.md-typeset details>.admonition-title:before,.md-typeset details>summary:before,.md-typeset summary:after{font-family:Material Icons;font-style:normal;font-variant:normal;font-weight:400;line-height:1;text-transform:none;white-space:nowrap;speak:none;word-wrap:normal;direction:ltr}.md-content__icon,.md-footer-nav__button,.md-header-nav__button,.md-nav__button,.md-nav__title:before,.md-search-result__article--document:before{display:inline-block;margin:.4rem;padding:.8rem;font-size:2.4rem;cursor:pointer}.md-icon--arrow-back:before{content:"\E5C4"}.md-icon--arrow-forward:before{content:"\E5C8"}.md-icon--menu:before{content:"\E5D2"}.md-icon--search:before{content:"\E8B6"}[dir=rtl] .md-icon--arrow-back:before{content:"\E5C8"}[dir=rtl] .md-icon--arrow-forward:before{content:"\E5C4"}body{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}body,input{color:rgba(0,0,0,.87);-webkit-font-feature-settings:"kern","liga";font-feature-settings:"kern","liga";font-family:Helvetica Neue,Helvetica,Arial,sans-serif}code,kbd,pre{color:rgba(0,0,0,.87);-webkit-font-feature-settings:"kern";font-feature-settings:"kern";font-family:Courier New,Courier,monospace}.md-typeset{font-size:1.6rem;line-height:1.6;-webkit-print-color-adjust:exact}.md-typeset blockquote,.md-typeset ol,.md-typeset p,.md-typeset ul{margin:1em 0}.md-typeset h1{margin:0 0 4rem;color:rgba(0,0,0,.54);font-size:3.125rem;line-height:1.3}.md-typeset h1,.md-typeset h2{font-weight:300;letter-spacing:-.01em}.md-typeset h2{margin:4rem 0 1.6rem;font-size:2.5rem;line-height:1.4}.md-typeset h3{margin:3.2rem 0 1.6rem;font-size:2rem;font-weight:400;letter-spacing:-.01em;line-height:1.5}.md-typeset h2+h3{margin-top:1.6rem}.md-typeset h4{font-size:1.6rem}.md-typeset h4,.md-typeset h5,.md-typeset h6{margin:1.6rem 0;font-weight:700;letter-spacing:-.01em}.md-typeset h5,.md-typeset h6{color:rgba(0,0,0,.54);font-size:1.28rem}.md-typeset h5{text-transform:uppercase}.md-typeset hr{margin:1.5em 0;border-bottom:.1rem dotted rgba(0,0,0,.26)}.md-typeset a{color:#3f51b5;word-break:break-word}.md-typeset a,.md-typeset a:before{transition:color .125s}.md-typeset a:active,.md-typeset a:hover{color:#536dfe}.md-typeset code,.md-typeset pre{background-color:hsla(0,0%,93%,.5);color:#37474f;font-size:85%;direction:ltr}.md-typeset code{margin:0 .29412em;padding:.07353em 0;border-radius:.2rem;box-shadow:.29412em 0 0 hsla(0,0%,93%,.5),-.29412em 0 0 hsla(0,0%,93%,.5);word-break:break-word;-webkit-box-decoration-break:clone;box-decoration-break:clone}.md-typeset h1 code,.md-typeset h2 code,.md-typeset h3 code,.md-typeset h4 code,.md-typeset h5 code,.md-typeset h6 code{margin:0;background-color:transparent;box-shadow:none}.md-typeset a>code{margin:inherit;padding:inherit;border-radius:none;background-color:inherit;color:inherit;box-shadow:none}.md-typeset pre{position:relative;margin:1em 0;border-radius:.2rem;line-height:1.4;-webkit-overflow-scrolling:touch}.md-typeset pre>code{display:block;margin:0;padding:1.05rem 1.2rem;background-color:transparent;font-size:inherit;box-shadow:none;-webkit-box-decoration-break:none;box-decoration-break:none;overflow:auto}.md-typeset pre>code::-webkit-scrollbar{width:.4rem;height:.4rem}.md-typeset pre>code::-webkit-scrollbar-thumb{background-color:rgba(0,0,0,.26)}.md-typeset pre>code::-webkit-scrollbar-thumb:hover{background-color:#536dfe}.md-typeset kbd{padding:0 .29412em;border:.1rem solid #c9c9c9;border-radius:.3rem;border-bottom-color:#bcbcbc;background-color:#fcfcfc;color:#555;font-size:85%;box-shadow:0 .1rem 0 #b0b0b0;word-break:break-word}.md-typeset mark{margin:0 .25em;padding:.0625em 0;border-radius:.2rem;background-color:rgba(255,235,59,.5);box-shadow:.25em 0 0 rgba(255,235,59,.5),-.25em 0 0 rgba(255,235,59,.5);word-break:break-word;-webkit-box-decoration-break:clone;box-decoration-break:clone}.md-typeset abbr{border-bottom:.1rem dotted rgba(0,0,0,.54);text-decoration:noneLine truncated
/*# sourceMappingURL=data:application/json;charset=utf-8;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbXSwibmFtZXMiOltdLCJtYXBwaW5ncyI6IiIsImZpbGUiOiJhc3NldHMvc3R5bGVzaGVldHMvYXBwbGljYXRpb24uNzhhYWIyZGMuY3NzIiwic291cmNlUm9vdCI6IiJ9*/
-2589
View File
File diff suppressed because it is too large. Load diff
-2315
View File
File diff suppressed because it is too large. Load diff
-584
View File
@@ -1,584 +0,0 @@
{
"docs": [
{
"location": "/",
"text": "LBRY JSON-RPC API Documentation\n\n\nblob_announce\n\n\nAnnounce blobs to the DHT\n\nArgs:\n 'blob_hash' : (str) announce a blob, specified by blob_hash\n 'stream_hash' : (str) announce all blobs associated with\n stream_hash\n 'sd_hash' : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful\n\n\n\n\nblob_availability\n\n\nGet blob availability\n\nArgs:\n 'blob_hash' : (str) check availability for this blob hash\n 'search_timeout' : (int) how long to search for peers for the blob\n in the dht\n 'blob_timeout' : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n \nis_available\n: \nbool, true if blob is available from a peer from peer list\n\n \nreachable_peers\n: [\nip\n:\nport\n],\n \nunreachable_peers\n: [\nip\n:\nport\n]\n }\n\n\n\n\nblob_delete\n\n\nDelete a blob\n\nArgs:\n 'blob_hash' : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message\n\n\n\n\nblob_get\n\n\nDownload and return a blob\n\nArgs:\n 'blob_hash' (required) : (str) blob hash of the blob to get\n 'timeout' : (int) timeout in number of seconds\n 'encoding' : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n 'payment_rate_manager' : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data\n\n\n\n\nblob_list\n\n\nReturns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nArgs:\n 'needed' : (bool) only return needed blobs\n 'finished' : (bool) only return finished blobs\n 'uri' : (str) filter blobs by stream in a uri\n 'stream_hash' : (str) filter blobs by stream hash\n 'sd_hash' : (str) filter blobs by sd hash\n 'page_size' : (int) results page size\n 'page' : (int) page of results to return\n\nReturns:\n (list) List of blob hashes\n\n\n\n\nblob_reflect\n\n\nReflects specified blobs\n\nArgs:\n 'reflector_server' : (str) reflector address\n\nReturns:\n (list) reflected blob hashes\n\n\n\n\nblob_reflect_all\n\n\nReflects all saved blobs\n\nArgs:\n None\n\nReturns:\n (bool) true if successful\n\n\n\n\nblock_show\n\n\nGet contents of a block\n\nArgs:\n 'blockhash' (required) : (str) hash of the block to look up\n 'height' (required) : (int) height of the block to look up\n\nReturns:\n (dict) Requested block\n\n\n\n\nchannel_export\n\n\nExport serialized channel signing information for a given certificate claim id\n\nArgs:\n 'claim_id' (required) : (str) Claim ID to export information about\n\nReturns:\n (str) Serialized certificate information\n\n\n\n\nchannel_import\n\n\nImport serialized channel signing information (to allow signing new claims to the channel)\n\nArgs:\n 'serialized_certificate_info' (required) : (str) certificate info\n\nReturns:\n (dict) Result dictionary\n\n\n\n\nchannel_list\n\n\nGet certificate claim infos for channels that can be published to\n\nArgs:\n None\n\nReturns:\n (list) ClaimDict, includes 'is_mine' field to indicate if the certificate claim\n is in the wallet.\n\n\n\n\nchannel_new\n\n\nGenerate a publisher key and create a new '@' prefixed certificate claim\n\nArgs:\n 'channel_name' (required) : (str) name of the channel prefixed with '@'\n 'amount' (required) : (float) bid amount on the channel\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }\n\n\n\n\nclaim_abandon\n\n\nAbandon a name and reclaim credits from the claim\n\nArgs:\n 'claim_id' : (str) claim_id of the claim to abandon\n 'txid' : (str) txid of the claim to abandon\n 'nout' : (int) nout of the claim to abandon\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n txid : (str) txid of resulting transaction\n fee : (float) fee paid for the transaction\n }\n\n\n\n\nclaim_list\n\n\nList current claims and information about them for a given name\n\nArgs:\n 'name' (required) : (str) name of the claim to list info about\n\nReturns:\n (dict) State Line truncated
"title": "API"
},
{
"location": "/#lbry-json-rpc-api-documentation",
"text": "",
"title": "LBRY JSON-RPC API Documentation"
},
{
"location": "/#blob_announce",
"text": "Announce blobs to the DHT\n\nArgs:\n 'blob_hash' : (str) announce a blob, specified by blob_hash\n 'stream_hash' : (str) announce all blobs associated with\n stream_hash\n 'sd_hash' : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful",
"title": "blob_announce"
},
{
"location": "/#blob_availability",
"text": "Get blob availability\n\nArgs:\n 'blob_hash' : (str) check availability for this blob hash\n 'search_timeout' : (int) how long to search for peers for the blob\n in the dht\n 'blob_timeout' : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n is_available : bool, true if blob is available from a peer from peer list \n reachable_peers : [ ip : port ],\n unreachable_peers : [ ip : port ]\n }",
"title": "blob_availability"
},
{
"location": "/#blob_delete",
"text": "Delete a blob\n\nArgs:\n 'blob_hash' : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message",
"title": "blob_delete"
},
{
"location": "/#blob_get",
"text": "Download and return a blob\n\nArgs:\n 'blob_hash' (required) : (str) blob hash of the blob to get\n 'timeout' : (int) timeout in number of seconds\n 'encoding' : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n 'payment_rate_manager' : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data",
"title": "blob_get"
},
{
"location": "/#blob_list",
"text": "Returns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nArgs:\n 'needed' : (bool) only return needed blobs\n 'finished' : (bool) only return finished blobs\n 'uri' : (str) filter blobs by stream in a uri\n 'stream_hash' : (str) filter blobs by stream hash\n 'sd_hash' : (str) filter blobs by sd hash\n 'page_size' : (int) results page size\n 'page' : (int) page of results to return\n\nReturns:\n (list) List of blob hashes",
"title": "blob_list"
},
{
"location": "/#blob_reflect",
"text": "Reflects specified blobs\n\nArgs:\n 'reflector_server' : (str) reflector address\n\nReturns:\n (list) reflected blob hashes",
"title": "blob_reflect"
},
{
"location": "/#blob_reflect_all",
"text": "Reflects all saved blobs\n\nArgs:\n None\n\nReturns:\n (bool) true if successful",
"title": "blob_reflect_all"
},
{
"location": "/#block_show",
"text": "Get contents of a block\n\nArgs:\n 'blockhash' (required) : (str) hash of the block to look up\n 'height' (required) : (int) height of the block to look up\n\nReturns:\n (dict) Requested block",
"title": "block_show"
},
{
"location": "/#channel_export",
"text": "Export serialized channel signing information for a given certificate claim id\n\nArgs:\n 'claim_id' (required) : (str) Claim ID to export information about\n\nReturns:\n (str) Serialized certificate information",
"title": "channel_export"
},
{
"location": "/#channel_import",
"text": "Import serialized channel signing information (to allow signing new claims to the channel)\n\nArgs:\n 'serialized_certificate_info' (required) : (str) certificate info\n\nReturns:\n (dict) Result dictionary",
"title": "channel_import"
},
{
"location": "/#channel_list",
"text": "Get certificate claim infos for channels that can be published to\n\nArgs:\n None\n\nReturns:\n (list) ClaimDict, includes 'is_mine' field to indicate if the certificate claim\n is in the wallet.",
"title": "channel_list"
},
{
"location": "/#channel_new",
"text": "Generate a publisher key and create a new '@' prefixed certificate claim\n\nArgs:\n 'channel_name' (required) : (str) name of the channel prefixed with '@'\n 'amount' (required) : (float) bid amount on the channel\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "channel_new"
},
{
"location": "/#claim_abandon",
"text": "Abandon a name and reclaim credits from the claim\n\nArgs:\n 'claim_id' : (str) claim_id of the claim to abandon\n 'txid' : (str) txid of the claim to abandon\n 'nout' : (int) nout of the claim to abandon\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n txid : (str) txid of resulting transaction\n fee : (float) fee paid for the transaction\n }",
"title": "claim_abandon"
},
{
"location": "/#claim_list",
"text": "List current claims and information about them for a given name\n\nArgs:\n 'name' (required) : (str) name of the claim to list info about\n\nReturns:\n (dict) State of claims assigned for the name\n {\n 'claims': (list) list of claims for the name\n [\n {\n 'amount': (float) amount assigned to the claim\n 'effective_amount': (float) total amount assigned to the claim,\n including supports\n 'claim_id': (str) claim ID of the claim\n 'height': (int) height of block containing the claim\n 'txid': (str) txid of the claim\n 'nout': (int) nout of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'supports': (list) a list of supports attached to the claim\n 'value': (str) the value of the claim\n },\n ]\n 'supports_without_claims': (list) supports without any claims attached to them\n 'last_takeover_height': (int) the height of last takeover for the name\n }",
"title": "claim_list"
},
{
"location": "/#claim_list_by_channel",
"text": "Get paginated claims in a channel specified by a channel uri\n\nArgs:\n 'uri' (required) : (str) uri of the channel\n 'uris' : (list) uris of the channel\n 'page' : (int) which page of results to return where page 1 is the first\n page, defaults to no pages\n 'page_size' : (int) number of results in a page, default of 10\n\nReturns:\n {\n resolved channel uri: {\n If there was an error:\n 'error': (str) error message\n\n 'claims_in_channel': the total number of results for the channel,\n\n If a page of results was requested:\n 'returned_page': page number returned,\n 'claims_in_channel': [\n {\n 'absolute_channel_position': (int) claim index number in sorted list of\n claims which assert to be part of the\n channel\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}],\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n ],\n }\n }",
"title": "claim_list_by_channel"
},
{
"location": "/#claim_list_mine",
"text": "List my name claims\n\nArgs:\n None\n\nReturns:\n (list) List of name claims owned by user\n [\n {\n 'address': (str) address that owns the claim\n 'amount': (float) amount assigned to the claim\n 'blocks_to_expiration': (int) number of blocks until it expires\n 'category': (str) claim , update , or support \n 'claim_id': (str) claim ID of the claim\n 'confirmations': (int) number of blocks of confirmations for the claim\n 'expiration_height': (int) the block height which the claim will expire\n 'expired': (bool) true if expired, false otherwise\n 'height': (int) height of the block containing the claim\n 'is_spent': (bool) true if claim is abandoned, false otherwise\n 'name': (str) name of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'txid': (str) txid of the claim\n 'nout': (int) nout of the claim\n 'value': (str) value of the claim\n },\n ]",
"title": "claim_list_mine"
},
{
"location": "/#claim_new_support",
"text": "Support a name claim\n\nArgs:\n 'name' (required) : (str) name of the claim to support\n 'claim_id' (required) : (str) claim_id of the claim to support\n 'amount' (required) : (float) amount of support\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"title": "claim_new_support"
},
{
"location": "/#claim_renew",
"text": "Renew claim(s) or support(s)\n\nArgs:\n 'outpoint' (required) : (str) outpoint of the claim to renew\n 'height' (required) : (str) update claims expiring before or at this block height\n\nReturns:\n (dict) Dictionary where key is the the original claim's outpoint and\n value is the result of the renewal\n {\n outpoint:{\n\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n },\n }",
"title": "claim_renew"
},
{
"location": "/#claim_send_to_address",
"text": "Send a name claim to an address\n\nArgs:\n 'claim_id' (required) : (str) claim_id to send\n 'address' (required) : (str) address to send the claim to\n 'amount' : (int) Amount of credits to claim name for, defaults to the current amount\n on the claim\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "claim_send_to_address"
},
{
"location": "/#claim_show",
"text": "Resolve claim info from txid/nout or with claim ID\n\nArgs:\n 'txid' : (str) look for claim with this txid, nout must\n also be specified\n 'nout' : (int) look for claim with this nout, txid must\n also be specified\n 'claim_id' : (str) look for claim with this claim id\n\nReturns:\n (dict) Dictionary containing claim info as below,\n\n {\n 'txid': (str) txid of claim\n 'nout': (int) nout of claim\n 'amount': (float) amount of claim\n 'value': (str) value of claim\n 'height' : (int) height of claim takeover\n 'claim_id': (str) claim ID of claim\n 'supports': (list) list of supports associated with claim\n }\n\n if claim cannot be resolved, dictionary as below will be returned\n\n {\n 'error': (str) reason for error\n }",
"title": "claim_show"
},
{
"location": "/#cli_test_command",
"text": "This command is only for testing the CLI argument parsing\nArgs:\n 'a_arg' : (bool) a arg\n 'b_arg' : (bool) b arg\n 'pos_arg' (required) : (int) pos arg\n 'pos_args' : (int) pos args\n 'pos_arg2' : (int) pos arg 2\n 'pos_arg3' : (int) pos arg 3\n\nReturns:\n pos args",
"title": "cli_test_command"
},
{
"location": "/#commands",
"text": "Return a list of available commands\n\nArgs:\n None\n\nReturns:\n (list) list of available commands",
"title": "commands"
},
{
"location": "/#daemon_stop",
"text": "Stop lbrynet-daemon\n\nArgs:\n None\n\nReturns:\n (string) Shutdown message",
"title": "daemon_stop"
},
{
"location": "/#file_delete",
"text": "Delete a LBRY file\n\nArgs:\n 'delete_from_download_dir' : (bool) delete file from download directory,\n instead of just deleting blobs\n 'delete_all' : (bool) if there are multiple matching files,\n allow the deletion of multiple files.\n Otherwise do not delete anything.\n 'sd_hash' : (str) delete by file sd hash\n 'file_name' : (str) delete by file name in downloads folder\n 'stream_hash' : (str) delete by file stream hash\n 'rowid' : (int) delete by file row id\n 'claim_id' : (str) delete by file claim id\n 'txid' : (str) delete by file claim txid\n 'nout' : (int) delete by file claim nout\n 'claim_name' : (str) delete by file claim name\n 'channel_claim_id' : (str) delete by file channel claim id\n 'channel_name' : (str) delete by file channel claim name\n\nReturns:\n (bool) true if deletion was successful",
"title": "file_delete"
},
{
"location": "/#file_list",
"text": "List files limited by optional filters\n\nArgs:\n 'sd_hash' : (str) get file with matching sd hash\n 'file_name' : (str) get file with matching file name in the\n downloads folder\n 'stream_hash' : (str) get file with matching stream hash\n 'rowid' : (int) get file with matching row id\n 'claim_id' : (str) get file with matching claim id\n 'outpoint' : (str) get file with matching claim outpoint\n 'txid' : (str) get file with matching claim txid\n 'nout' : (int) get file with matching claim nout\n 'channel_claim_id' : (str) get file with matching channel claim id\n 'channel_name' : (str) get file with matching channel name\n 'claim_name' : (str) get file with matching claim name\n 'full_status' : (bool) full status, populate the\n 'message' and 'size' fields\n 'sort' : (str) sort by any property, like 'file_name'\n or 'metadata.author'; to specify direction\n append ',asc' or ',desc'\n\nReturns:\n (list) List of files\n\n [\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) None if full_status is false or if claim is not found,\n 'outpoint': (str) None if full_status is false or if claim is not found,\n 'txid': (str) None if full_status is false or if claim is not found,\n 'nout': (int) None if full_status is false or if claim is not found,\n 'metadata': (dict) None if full_status is false or if claim is not found,\n 'channel_claim_id': (str) None if full_status is false or if claim is not found or signed,\n 'channel_name': (str) None if full_status is false or if claim is not found or signed,\n 'claim_name': (str) None if full_status is false or if claim is not found\n },\n ]",
"title": "file_list"
},
{
"location": "/#file_reflect",
"text": "Reflect all the blobs in a file matching the filter criteria\n\nArgs:\n 'sd_hash' : (str) get file with matching sd hash\n 'file_name' : (str) get file with matching file name in the\n downloads folder\n 'stream_hash' : (str) get file with matching stream hash\n 'rowid' : (int) get file with matching row id\n 'reflector' : (str) reflector server, ip address or url\n by default choose a server from the config\n\nReturns:\n (list) list of blobs reflected",
"title": "file_reflect"
},
{
"location": "/#file_set_status",
"text": "Start or stop downloading a file\n\nArgs:\n 'status' (required) : (str) one of start or stop \n 'sd_hash' : (str) set status of file with matching sd hash\n 'file_name' : (str) set status of file with matching file name in the\n downloads folder\n 'stream_hash' : (str) set status of file with matching stream hash\n 'rowid' : (int) set status of file with matching row id\n\nReturns:\n (str) Confirmation message",
"title": "file_set_status"
},
{
"location": "/#get",
"text": "Download stream from a LBRY name.\n\nArgs:\n 'uri' : (str) uri of the content to download\n 'file_name' : (str) specified name for the downloaded file\n 'timeout' : (int) download timeout in number of seconds\n\nReturns:\n (dict) Dictionary containing information about the stream\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) claim id,\n 'outpoint': (str) claim outpoint string,\n 'txid': (str) claim txid,\n 'nout': (int) claim nout,\n 'metadata': (dict) claim metadata,\n 'channel_claim_id': (str) None if claim is not signed\n 'channel_name': (str) None if claim is not signed\n 'claim_name': (str) claim name\n }",
"title": "get"
},
{
"location": "/#help",
"text": "Return a useful message for an API command\n\nArgs:\n 'command' : (str) command to retrieve documentation for\n\nReturns:\n (str) Help message",
"title": "help"
},
{
"location": "/#peer_list",
"text": "Get peers for blob hash\n\nArgs:\n 'blob_hash' (required) : (str) find available peers for this blob hash\n 'timeout' : (int) peer search timeout in seconds\n\nReturns:\n (list) List of contact dictionaries {'host': peer ip , 'port': peer port , 'node_id': peer node id }",
"title": "peer_list"
},
{
"location": "/#peer_ping",
"text": "Send a kademlia ping to the specified peer. If address and port are provided the peer is directly pinged,\nif not provided the peer is located first.\n\nArgs:\n 'address' : (str) ip address of the peer\n 'port' : (int) udp port of the peer\n\nReturns:\n (str) pong, or {'error': error message } if an error is encountered",
"title": "peer_ping"
},
{
"location": "/#publish",
"text": "Make a new name claim and publish associated data to lbrynet,\nupdate over existing claim if user already has a claim for name.\n\nFields required in the final Metadata are:\n 'title'\n 'description'\n 'author'\n 'language'\n 'license'\n 'nsfw'\n\nMetadata can be set by either using the metadata argument or by setting individual arguments\nfee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\nor sources. Individual arguments will overwrite the fields specified in metadata argument.\n\nArgs:\n 'name' (required) : (str) name of the content\n 'bid' (required) : (decimal) amount to back the claim\n 'metadata' : (dict) ClaimDict to associate with the claim.\n 'file_path' : (str) path to file to be associated with name. If provided,\n a lbry stream of this file will be used in 'sources'.\n If no path is given but a sources dict is provided,\n it will be used. If neither are provided, an\n error is raised.\n 'fee' : (dict) Dictionary representing key fee to download content:\n {\n 'currency': currency_symbol,\n 'amount': decimal,\n 'address': str, optional\n }\n supported currencies: LBC, USD, BTC\n If an address is not provided a new one will be\n automatically generated. Default fee is zero.\n 'title' : (str) title of the publication\n 'description' : (str) description of the publication\n 'author' : (str) author of the publication\n 'language' : (str) language of the publication\n 'license' : (str) publication license\n 'license_url' : (str) publication license url\n 'thumbnail' : (str) thumbnail url\n 'preview' : (str) preview url\n 'nsfw' : (bool) whether the content is nsfw\n 'sources' : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file\n 'channel_name' : (str) name of the publisher channel name in the wallet\n 'channel_id' : (str) claim id of the publisher channel, does not check\n for channel claim being in the wallet. This allows\n publishing to a channel where only the certificate\n private key is in the wallet.\n 'claim_address' : (str) address where the claim is sent to, if not specified\n new address will automatically be created\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (decimal) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "publish"
},
{
"location": "/#report_bug",
"text": "Report a bug to slack\n\nArgs:\n 'message' (required) : (str) Description of the bug\n\nReturns:\n (bool) true if successful",
"title": "report_bug"
},
{
"location": "/#resolve",
"text": "Resolve given LBRY URIs\n\nArgs:\n 'force' : (bool) force refresh and ignore cache\n 'uri' (required) : (str) uri to resolve\n 'uris' : (list) uris to resolve\n\nReturns:\n Dictionary of results, keyed by uri\n ' uri ': {\n If a resolution error occurs:\n 'error': Error message\n\n If the uri resolves to a channel or a claim in a channel:\n 'certificate': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the certificate claim,\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}],\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n\n If the uri resolves to a channel:\n 'claims_in_channel': (int) number of claims in the channel,\n\n If the uri resolves to a claim:\n 'claim': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the claim,\n 'channel_name': (str) channel name if claim is in a channel\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}]\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n }",
"title": "resolve"
},
{
"location": "/#resolve_name",
"text": "Resolve stream info from a LBRY name\n\nArgs:\n 'name' (required) : (str) the name to resolve\n 'force' : (bool) force refresh and do not check cache\n\nReturns:\n (dict) Metadata dictionary from name claim, None if the name is not\n resolvable",
"title": "resolve_name"
},
{
"location": "/#routing_table_get",
"text": "Get DHT routing information\n\nArgs:\n None\n\nReturns:\n (dict) dictionary containing routing and contact information\n {\n buckets : {\n bucket index : [\n {\n address : (str) peer address,\n port : (int) peer udp port\n node_id : (str) peer node id,\n blobs : (list) blob hashes announced by peer\n }\n ]\n },\n contacts : (list) contact node ids,\n blob_hashes : (list) all of the blob hashes stored by peers in the list of buckets,\n node_id : (str) the local dht node id\n }",
"title": "routing_table_get"
},
{
"location": "/#settings_get",
"text": "Get daemon settings\n\nArgs:\n None\n\nReturns:\n (dict) Dictionary of daemon settings\n See ADJUSTABLE_SETTINGS in lbrynet/conf.py for full list of settings",
"title": "settings_get"
},
{
"location": "/#settings_set",
"text": "Set daemon settings\n\nArgs:\n 'download_directory' : (str) path of download directory\n 'data_rate' : (float) 0.0001\n 'download_timeout' : (int) 180\n 'peer_port' : (int) 3333\n 'max_key_fee' : (dict) maximum key fee for downloads,\n in the format:\n {\n 'currency': currency_symbol ,\n 'amount': amount \n }.\n In the CLI, it must be an escaped JSON string\n Supported currency symbols: LBC, USD, BTC\n 'disable_max_key_fee' : (bool) False\n 'use_upnp' : (bool) True\n 'run_reflector_server' : (bool) False\n 'cache_time' : (int) 150\n 'reflect_uploads' : (bool) True\n 'share_usage_data' : (bool) True\n 'peer_search_timeout' : (int) 3\n 'sd_download_timeout' : (int) 3\n 'auto_renew_claim_height_delta' : (int) 0\n claims set to expire within this many blocks will be\n automatically renewed after startup (if set to 0, renews\n will not be made automatically)\n\nReturns:\n (dict) Updated dictionary of daemon settings",
"title": "settings_set"
},
{
"location": "/#status",
"text": "Get daemon status\n\nArgs:\n None\n\nReturns:\n (dict) lbrynet-daemon status\n {\n 'installation_id': (str) installation id - base58,\n 'is_running': (bool),\n 'is_first_run': bool,\n 'skipped_components': (list) [names of skipped components (str)],\n 'startup_status': { Does not include components which have been skipped\n 'database': (bool),\n 'wallet': (bool),\n 'session': (bool),\n 'dht': (bool),\n 'hash_announcer': (bool),\n 'stream_identifier': (bool),\n 'file_manager': (bool),\n 'blob_manager': (bool),\n 'blockchain_headers': (bool),\n 'peer_protocol_server': (bool),\n 'reflector': (bool),\n 'upnp': (bool),\n 'exchange_rate_manager': (bool),\n },\n 'connection_status': {\n 'code': (str) connection status code,\n 'message': (str) connection status message\n },\n 'blockchain_headers': {\n 'downloading_headers': (bool),\n 'download_progress': (float) 0-100.0\n },\n 'wallet': {\n 'blocks': (int) local blockchain height,\n 'blocks_behind': (int) remote_height - local_height,\n 'best_blockhash': (str) block hash of most recent block,\n 'is_encrypted': (bool),\n 'is_locked': (bool),\n },\n 'dht': {\n 'node_id': (str) lbry dht node id - hex encoded,\n 'peers_in_routing_table': (int) the number of peers in the routing table,\n },\n 'blob_manager': {\n 'finished_blobs': (int) number of finished blobs in the blob manager,\n },\n 'hash_announcer': {\n 'announce_queue_size': (int) number of blobs currently queued to be announced\n },\n 'file_manager': {\n 'managed_files': (int) count of files in the file manager,\n }\n }",
"title": "status"
},
{
"location": "/#stream_availability",
"text": "Get stream availability for lbry uri\n\nArgs:\n 'uri' (required) : (str) check availability for this uri\n 'search_timeout' : (int) how long to search for peers for the blob\n in the dht\n 'blob_timeout' : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n 'is_available': bool ,\n 'did_decode': bool ,\n 'did_resolve': bool ,\n 'is_stream': bool ,\n 'num_blobs_in_stream': int ,\n 'sd_hash': str ,\n 'sd_blob_availability': dict see `blob_availability`,\n 'head_blob_hash': str ,\n 'head_blob_availability': dict see `blob_availability`,\n 'use_upnp': bool ,\n 'upnp_redirect_is_set': bool ,\n 'error': None | str error message\n }",
"title": "stream_availability"
},
{
"location": "/#stream_cost_estimate",
"text": "Get estimated cost for a lbry stream\n\nArgs:\n 'uri' (required) : (str) uri to use\n 'size' : (float) stream size in bytes. if provided an sd blob won't be\n downloaded.\n\nReturns:\n (float) Estimated cost in lbry credits, returns None if uri is not\n resolvable",
"title": "stream_cost_estimate"
},
{
"location": "/#transaction_list",
"text": "List transactions belonging to wallet\n\nArgs:\n None\n\nReturns:\n (list) List of transactions\n\n {\n claim_info : (list) claim info if in txn [{\n address : (str) address of claim,\n balance_delta : (float) bid amount,\n amount : (float) claim amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n nout : (int) nout\n }],\n abandon_info : (list) abandon info if in txn [{\n address : (str) address of abandoned claim,\n balance_delta : (float) returned amount,\n amount : (float) claim amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n nout : (int) nout\n }],\n confirmations : (int) number of confirmations for the txn,\n date : (str) date and time of txn,\n fee : (float) txn fee,\n support_info : (list) support info if in txn [{\n address : (str) address of support,\n balance_delta : (float) support amount,\n amount : (float) support amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n is_tip : (bool),\n nout : (int) nout\n }],\n timestamp : (int) timestamp,\n txid : (str) txn id,\n update_info : (list) update info if in txn [{\n address : (str) address of claim,\n balance_delta : (float) credited/debited\n amount : (float) absolute amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n nout : (int) nout\n }],\n value : (float) value of txn\n }",
"title": "transaction_list"
},
{
"location": "/#transaction_show",
"text": "Get a decoded transaction from a txid\n\nArgs:\n 'txid' (required) : (str) txid of the transaction\n\nReturns:\n (dict) JSON formatted transaction",
"title": "transaction_show"
},
{
"location": "/#utxo_list",
"text": "List unspent transaction outputs\n\nArgs:\n None\n\nReturns:\n (list) List of unspent transaction outputs (UTXOs)\n [\n {\n address : (str) the output address\n amount : (float) unspent amount\n height : (int) block height\n is_claim : (bool) is the tx a claim\n is_coinbase : (bool) is the tx a coinbase tx\n is_support : (bool) is the tx a support\n is_update : (bool) is the tx an update\n nout : (int) nout of the output\n txid : (str) txid of the output\n },\n ...\n ]",
"title": "utxo_list"
},
{
"location": "/#version",
"text": "Get lbry version information\n\nArgs:\n None\n\nReturns:\n (dict) Dictionary of lbry version information\n {\n 'build': (str) build type (e.g. dev , rc , release ),\n 'ip': (str) remote ip, if available,\n 'lbrynet_version': (str) lbrynet_version,\n 'lbryum_version': (str) lbryum_version,\n 'lbryschema_version': (str) lbryschema_version,\n 'os_release': (str) os release string\n 'os_system': (str) os name\n 'platform': (str) platform string\n 'processor': (str) processor type,\n 'python_version': (str) python version,\n }",
"title": "version"
},
{
"location": "/#wallet_balance",
"text": "Return the balance of the wallet\n\nArgs:\n 'address' : (str) If provided only the balance for this\n address will be given\n 'include_unconfirmed' : (bool) Include unconfirmed\n\nReturns:\n (float) amount of lbry credits in wallet",
"title": "wallet_balance"
},
{
"location": "/#wallet_decrypt",
"text": "Decrypt an encrypted wallet, this will remove the wallet password\n\nArgs:\n None\n\nReturns:\n (bool) true if wallet is decrypted, otherwise false",
"title": "wallet_decrypt"
},
{
"location": "/#wallet_encrypt",
"text": "Encrypt a wallet with a password, if the wallet is already encrypted this will update\nthe password\n\nArgs:\n 'new_password' (required) : (str) password string to be used for encrypting wallet\n\nReturns:\n (bool) true if wallet is decrypted, otherwise false",
"title": "wallet_encrypt"
},
{
"location": "/#wallet_is_address_mine",
"text": "Checks if an address is associated with the current wallet.\n\nArgs:\n 'address' (required) : (str) address to check\n\nReturns:\n (bool) true, if address is associated with current wallet",
"title": "wallet_is_address_mine"
},
{
"location": "/#wallet_list",
"text": "List wallet addresses\n\nArgs:\n None\n\nReturns:\n List of wallet addresses",
"title": "wallet_list"
},
{
"location": "/#wallet_new_address",
"text": "Generate a new wallet address\n\nArgs:\n None\n\nReturns:\n (str) New wallet address in base58",
"title": "wallet_new_address"
},
{
"location": "/#wallet_prefill_addresses",
"text": "Create new addresses, each containing `amount` credits\n\nArgs:\n 'no_broadcast' : (bool) whether to broadcast or not\n 'num_addresses' (required) : (int) num of addresses to create\n 'amount' (required) : (float) initial amount in each address\n\nReturns:\n (dict) the resulting transaction",
"title": "wallet_prefill_addresses"
},
{
"location": "/#wallet_public_key",
"text": "Get public key from wallet address\n\nArgs:\n 'address' (required) : (str) address for which to get the public key\n\nReturns:\n (list) list of public keys associated with address.\n Could contain more than one public key if multisig.",
"title": "wallet_public_key"
},
{
"location": "/#wallet_send",
"text": "Send credits. If given an address, send credits to it. If given a claim id, send a tip\nto the owner of a claim specified by uri. A tip is a claim support where the recipient\nof the support is the claim address for the claim being supported.\n\nArgs:\n 'amount' (required) : (Decimal) amount of credit to send\n 'address' (required) : (str) address to send credits to\n 'claim_id' (required) : (str) claim_id of the claim to send to tip to\n\nReturns:\n If sending to an address:\n (bool) true if payment successfully scheduled\n\n If sending a claim tip:\n (dict) Dictionary containing the result of the support\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"title": "wallet_send"
},
{
"location": "/#wallet_unlock",
"text": "Unlock an encrypted wallet\n\nArgs:\n 'password' (required) : (str) password for unlocking wallet\n\nReturns:\n (bool) true if wallet is unlocked, otherwise false",
"title": "wallet_unlock"
},
{
"location": "/#wallet_unused_address",
"text": "Return an address containing no balance, will create\na new address if there is none.\n\nArgs:\n None\n\nReturns:\n (str) Unused wallet address in base58",
"title": "wallet_unused_address"
},
{
"location": "/cli/",
"text": "LBRY JSON-RPC API Documentation\n\n\nblob_announce\n\n\nAnnounce blobs to the DHT\n\nUsage:\n blob_announce [\nblob_hash\n | --blob_hash=\nblob_hash\n]\n [\nstream_hash\n | --stream_hash=\nstream_hash\n] | [\nsd_hash\n | --sd_hash=\nsd_hash\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) announce a blob, specified by blob_hash\n --stream_hash=\nstream_hash\n : (str) announce all blobs associated with\n stream_hash\n --sd_hash=\nsd_hash\n : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful\n\n\n\n\nblob_availability\n\n\nGet blob availability\n\nUsage:\n blob_availability (\nblob_hash\n) [\nsearch_timeout\n | --search_timeout=\nsearch_timeout\n]\n [\nblob_timeout\n | --blob_timeout=\nblob_timeout\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) check availability for this blob hash\n --search_timeout=\nsearch_timeout\n : (int) how long to search for peers for the blob\n in the dht\n --blob_timeout=\nblob_timeout\n : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n \nis_available\n: \nbool, true if blob is available from a peer from peer list\n\n \nreachable_peers\n: [\nip\n:\nport\n],\n \nunreachable_peers\n: [\nip\n:\nport\n]\n }\n\n\n\n\nblob_delete\n\n\nDelete a blob\n\nUsage:\n blob_delete (\nblob_hash\n | --blob_hash=\nblob_hash)\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message\n\n\n\n\nblob_get\n\n\nDownload and return a blob\n\nUsage:\n blob_get (\nblob_hash\n | --blob_hash=\nblob_hash\n) [--timeout=\ntimeout\n]\n [--encoding=\nencoding\n] [--payment_rate_manager=\npayment_rate_manager\n]\n\n\nOptions:\n --blob_hash=\nblob_hash\n : (str) blob hash of the blob to get\n --timeout=\ntimeout\n : (int) timeout in number of seconds\n --encoding=\nencoding\n : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n --payment_rate_manager=\npayment_rate_manager\n : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data\n\n\n\n\nblob_list\n\n\nReturns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nUsage:\n blob_list [--needed] [--finished] [\nuri\n | --uri=\nuri\n]\n [\nstream_hash\n | --stream_hash=\nstream_hash\n]\n [\nsd_hash\n | --sd_hash=\nsd_hash\n]\n [\npage_size\n | --page_size=\npage_size\n]\n [\npage\n | --page=\npage\n]\n\n\nOptions:\n --needed : (bool) only return needed blobs\n --finished : (bool) only return finished blobs\n --uri=\nuri\n : (str) filter blobs by stream in a uri\n --stream_hash=\nstream_hash\n : (str) filter blobs by stream hash\n --sd_hash=\nsd_hash\n : (str) filter blobs by sd hash\n --page_size=\npage_size\n : (int) results page size\n --page=\npage\n : (int) page of results to return\n\nReturns:\n (list) List of blob hashes\n\n\n\n\nblob_reflect\n\n\nReflects specified blobs\n\nUsage:\n blob_reflect (\nblob_hashes\n...) [--reflector_server=\nreflector_server\n]\n\n\nOptions:\n --reflector_server=\nreflector_server\n : (str) reflector address\n\nReturns:\n (list) reflected blob hashes\n\n\n\n\nblob_reflect_all\n\n\nReflects all saved blobs\n\nUsage:\n blob_reflect_all\n\n\nOptions:\n None\n\nReturns:\n (bool) true if successful\n\n\n\n\nblock_show\n\n\nGet contents of a block\n\nUsage:\n block_show (\nblockhash\n | --blockhash=\nblockhash\n) | (\nheight\n | --height=\nheight\n)\n\n\nOptions:\n --blockhash=\nblockhash\n : (str) hash of the block to look up\n --height=\nheight\n : (int) height of the block to look up\n\nReturns:\n (dict) Requested block\n\n\n\n\nchannel_export\n\n\nExport serialized channel signing information for a given certificate claim id\n\nUsage:\n channel_export (\nclaim_id\n | --claim_id=\nclaim_id\n)\n\n\nOptions:\n --claim_id=\nclaim_id\n : (str) Claim ID to export informatiLine truncated
"title": "CLI"
},
{
"location": "/cli/#lbry-json-rpc-api-documentation",
"text": "",
"title": "LBRY JSON-RPC API Documentation"
},
{
"location": "/cli/#blob_announce",
"text": "Announce blobs to the DHT\n\nUsage:\n blob_announce [ blob_hash | --blob_hash= blob_hash ]\n [ stream_hash | --stream_hash= stream_hash ] | [ sd_hash | --sd_hash= sd_hash ]\n\n\nOptions:\n --blob_hash= blob_hash : (str) announce a blob, specified by blob_hash\n --stream_hash= stream_hash : (str) announce all blobs associated with\n stream_hash\n --sd_hash= sd_hash : (str) announce all blobs associated with\n sd_hash and the sd_hash itself\n\nReturns:\n (bool) true if successful",
"title": "blob_announce"
},
{
"location": "/cli/#blob_availability",
"text": "Get blob availability\n\nUsage:\n blob_availability ( blob_hash ) [ search_timeout | --search_timeout= search_timeout ]\n [ blob_timeout | --blob_timeout= blob_timeout ]\n\n\nOptions:\n --blob_hash= blob_hash : (str) check availability for this blob hash\n --search_timeout= search_timeout : (int) how long to search for peers for the blob\n in the dht\n --blob_timeout= blob_timeout : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n is_available : bool, true if blob is available from a peer from peer list \n reachable_peers : [ ip : port ],\n unreachable_peers : [ ip : port ]\n }",
"title": "blob_availability"
},
{
"location": "/cli/#blob_delete",
"text": "Delete a blob\n\nUsage:\n blob_delete ( blob_hash | --blob_hash= blob_hash)\n\n\nOptions:\n --blob_hash= blob_hash : (str) blob hash of the blob to delete\n\nReturns:\n (str) Success/fail message",
"title": "blob_delete"
},
{
"location": "/cli/#blob_get",
"text": "Download and return a blob\n\nUsage:\n blob_get ( blob_hash | --blob_hash= blob_hash ) [--timeout= timeout ]\n [--encoding= encoding ] [--payment_rate_manager= payment_rate_manager ]\n\n\nOptions:\n --blob_hash= blob_hash : (str) blob hash of the blob to get\n --timeout= timeout : (int) timeout in number of seconds\n --encoding= encoding : (str) by default no attempt at decoding\n is made, can be set to one of the\n following decoders:\n 'json'\n --payment_rate_manager= payment_rate_manager : (str) if not given the default payment rate\n manager will be used.\n supported alternative rate managers:\n 'only-free'\n\nReturns:\n (str) Success/Fail message or (dict) decoded data",
"title": "blob_get"
},
{
"location": "/cli/#blob_list",
"text": "Returns blob hashes. If not given filters, returns all blobs known by the blob manager\n\nUsage:\n blob_list [--needed] [--finished] [ uri | --uri= uri ]\n [ stream_hash | --stream_hash= stream_hash ]\n [ sd_hash | --sd_hash= sd_hash ]\n [ page_size | --page_size= page_size ]\n [ page | --page= page ]\n\n\nOptions:\n --needed : (bool) only return needed blobs\n --finished : (bool) only return finished blobs\n --uri= uri : (str) filter blobs by stream in a uri\n --stream_hash= stream_hash : (str) filter blobs by stream hash\n --sd_hash= sd_hash : (str) filter blobs by sd hash\n --page_size= page_size : (int) results page size\n --page= page : (int) page of results to return\n\nReturns:\n (list) List of blob hashes",
"title": "blob_list"
},
{
"location": "/cli/#blob_reflect",
"text": "Reflects specified blobs\n\nUsage:\n blob_reflect ( blob_hashes ...) [--reflector_server= reflector_server ]\n\n\nOptions:\n --reflector_server= reflector_server : (str) reflector address\n\nReturns:\n (list) reflected blob hashes",
"title": "blob_reflect"
},
{
"location": "/cli/#blob_reflect_all",
"text": "Reflects all saved blobs\n\nUsage:\n blob_reflect_all\n\n\nOptions:\n None\n\nReturns:\n (bool) true if successful",
"title": "blob_reflect_all"
},
{
"location": "/cli/#block_show",
"text": "Get contents of a block\n\nUsage:\n block_show ( blockhash | --blockhash= blockhash ) | ( height | --height= height )\n\n\nOptions:\n --blockhash= blockhash : (str) hash of the block to look up\n --height= height : (int) height of the block to look up\n\nReturns:\n (dict) Requested block",
"title": "block_show"
},
{
"location": "/cli/#channel_export",
"text": "Export serialized channel signing information for a given certificate claim id\n\nUsage:\n channel_export ( claim_id | --claim_id= claim_id )\n\n\nOptions:\n --claim_id= claim_id : (str) Claim ID to export information about\n\nReturns:\n (str) Serialized certificate information",
"title": "channel_export"
},
{
"location": "/cli/#channel_import",
"text": "Import serialized channel signing information (to allow signing new claims to the channel)\n\nUsage:\n channel_import ( serialized_certificate_info | --serialized_certificate_info= serialized_certificate_info )\n\n\nOptions:\n --serialized_certificate_info= serialized_certificate_info : (str) certificate info\n\nReturns:\n (dict) Result dictionary",
"title": "channel_import"
},
{
"location": "/cli/#channel_list",
"text": "Get certificate claim infos for channels that can be published to\n\nUsage:\n channel_list\n\n\nOptions:\n None\n\nReturns:\n (list) ClaimDict, includes 'is_mine' field to indicate if the certificate claim\n is in the wallet.",
"title": "channel_list"
},
{
"location": "/cli/#channel_new",
"text": "Generate a publisher key and create a new '@' prefixed certificate claim\n\nUsage:\n channel_new ( channel_name | --channel_name= channel_name )\n ( amount | --amount= amount )\n\n\nOptions:\n --channel_name= channel_name : (str) name of the channel prefixed with '@'\n --amount= amount : (float) bid amount on the channel\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "channel_new"
},
{
"location": "/cli/#claim_abandon",
"text": "Abandon a name and reclaim credits from the claim\n\nUsage:\n claim_abandon [ claim_id | --claim_id= claim_id ]\n [ txid | --txid= txid ] [ nout | --nout= nout ]\n\n\nOptions:\n --claim_id= claim_id : (str) claim_id of the claim to abandon\n --txid= txid : (str) txid of the claim to abandon\n --nout= nout : (int) nout of the claim to abandon\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n txid : (str) txid of resulting transaction\n fee : (float) fee paid for the transaction\n }",
"title": "claim_abandon"
},
{
"location": "/cli/#claim_list",
"text": "List current claims and information about them for a given name\n\nUsage:\n claim_list ( name | --name= name )\n\n\nOptions:\n --name= name : (str) name of the claim to list info about\n\nReturns:\n (dict) State of claims assigned for the name\n {\n 'claims': (list) list of claims for the name\n [\n {\n 'amount': (float) amount assigned to the claim\n 'effective_amount': (float) total amount assigned to the claim,\n including supports\n 'claim_id': (str) claim ID of the claim\n 'height': (int) height of block containing the claim\n 'txid': (str) txid of the claim\n 'nout': (int) nout of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'supports': (list) a list of supports attached to the claim\n 'value': (str) the value of the claim\n },\n ]\n 'supports_without_claims': (list) supports without any claims attached to them\n 'last_takeover_height': (int) the height of last takeover for the name\n }",
"title": "claim_list"
},
{
"location": "/cli/#claim_list_by_channel",
"text": "Get paginated claims in a channel specified by a channel uri\n\nUsage:\n claim_list_by_channel ( uri | --uri= uri ) [ uris ...] [--page= page ]\n [--page_size= page_size ]\n\n\nOptions:\n --uri= uri : (str) uri of the channel\n --uris= uris : (list) uris of the channel\n --page= page : (int) which page of results to return where page 1 is the first\n page, defaults to no pages\n --page_size= page_size : (int) number of results in a page, default of 10\n\nReturns:\n {\n resolved channel uri: {\n If there was an error:\n 'error': (str) error message\n\n 'claims_in_channel': the total number of results for the channel,\n\n If a page of results was requested:\n 'returned_page': page number returned,\n 'claims_in_channel': [\n {\n 'absolute_channel_position': (int) claim index number in sorted list of\n claims which assert to be part of the\n channel\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}],\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n ],\n }\n }",
"title": "claim_list_by_channel"
},
{
"location": "/cli/#claim_list_mine",
"text": "List my name claims\n\nUsage:\n claim_list_mine\n\n\nOptions:\n None\n\nReturns:\n (list) List of name claims owned by user\n [\n {\n 'address': (str) address that owns the claim\n 'amount': (float) amount assigned to the claim\n 'blocks_to_expiration': (int) number of blocks until it expires\n 'category': (str) claim , update , or support \n 'claim_id': (str) claim ID of the claim\n 'confirmations': (int) number of blocks of confirmations for the claim\n 'expiration_height': (int) the block height which the claim will expire\n 'expired': (bool) true if expired, false otherwise\n 'height': (int) height of the block containing the claim\n 'is_spent': (bool) true if claim is abandoned, false otherwise\n 'name': (str) name of the claim\n 'permanent_url': (str) permanent url of the claim,\n 'txid': (str) txid of the claim\n 'nout': (int) nout of the claim\n 'value': (str) value of the claim\n },\n ]",
"title": "claim_list_mine"
},
{
"location": "/cli/#claim_new_support",
"text": "Support a name claim\n\nUsage:\n claim_new_support ( name | --name= name ) ( claim_id | --claim_id= claim_id )\n ( amount | --amount= amount )\n\n\nOptions:\n --name= name : (str) name of the claim to support\n --claim_id= claim_id : (str) claim_id of the claim to support\n --amount= amount : (float) amount of support\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"title": "claim_new_support"
},
{
"location": "/cli/#claim_renew",
"text": "Renew claim(s) or support(s)\n\nUsage:\n claim_renew ( outpoint | --outpoint= outpoint ) | ( height | --height= height )\n\n\nOptions:\n --outpoint= outpoint : (str) outpoint of the claim to renew\n --height= height : (str) update claims expiring before or at this block height\n\nReturns:\n (dict) Dictionary where key is the the original claim's outpoint and\n value is the result of the renewal\n {\n outpoint:{\n\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n },\n }",
"title": "claim_renew"
},
{
"location": "/cli/#claim_send_to_address",
"text": "Send a name claim to an address\n\nUsage:\n claim_send_to_address ( claim_id | --claim_id= claim_id )\n ( address | --address= address )\n [ amount | --amount= amount ]\n\n\nOptions:\n --claim_id= claim_id : (str) claim_id to send\n --address= address : (str) address to send the claim to\n --amount amount : (int) Amount of credits to claim name for, defaults to the current amount\n on the claim\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (float) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "claim_send_to_address"
},
{
"location": "/cli/#claim_show",
"text": "Resolve claim info from txid/nout or with claim ID\n\nUsage:\n claim_show [ txid | --txid= txid ] [ nout | --nout= nout ]\n [ claim_id | --claim_id= claim_id ]\n\n\nOptions:\n --txid= txid : (str) look for claim with this txid, nout must\n also be specified\n --nout= nout : (int) look for claim with this nout, txid must\n also be specified\n --claim_id= claim_id : (str) look for claim with this claim id\n\nReturns:\n (dict) Dictionary containing claim info as below,\n\n {\n 'txid': (str) txid of claim\n 'nout': (int) nout of claim\n 'amount': (float) amount of claim\n 'value': (str) value of claim\n 'height' : (int) height of claim takeover\n 'claim_id': (str) claim ID of claim\n 'supports': (list) list of supports associated with claim\n }\n\n if claim cannot be resolved, dictionary as below will be returned\n\n {\n 'error': (str) reason for error\n }",
"title": "claim_show"
},
{
"location": "/cli/#cli_test_command",
"text": "This command is only for testing the CLI argument parsing\nUsage:\n cli_test_command [--a_arg] [--b_arg] ( pos_arg | --pos_arg= pos_arg )\n [ pos_args ...] [--pos_arg2= pos_arg2 ]\n [--pos_arg3= pos_arg3 ]\n\n\nOptions:\n --a_arg : (bool) a arg\n --b_arg : (bool) b arg\n --pos_arg= pos_arg : (int) pos arg\n --pos_args= pos_args : (int) pos args\n --pos_arg2= pos_arg2 : (int) pos arg 2\n --pos_arg3= pos_arg3 : (int) pos arg 3\n\nReturns:\n pos args",
"title": "cli_test_command"
},
{
"location": "/cli/#commands",
"text": "Return a list of available commands\n\nUsage:\n commands\n\n\nOptions:\n None\n\nReturns:\n (list) list of available commands",
"title": "commands"
},
{
"location": "/cli/#daemon_stop",
"text": "Stop lbrynet-daemon\n\nUsage:\n daemon_stop\n\n\nOptions:\n None\n\nReturns:\n (string) Shutdown message",
"title": "daemon_stop"
},
{
"location": "/cli/#file_delete",
"text": "Delete a LBRY file\n\nUsage:\n file_delete [--delete_from_download_dir] [--delete_all] [--sd_hash= sd_hash ] [--file_name= file_name ]\n [--stream_hash= stream_hash ] [--rowid= rowid ] [--claim_id= claim_id ] [--txid= txid ]\n [--nout= nout ] [--claim_name= claim_name ] [--channel_claim_id= channel_claim_id ]\n [--channel_name= channel_name ]\n\n\nOptions:\n --delete_from_download_dir : (bool) delete file from download directory,\n instead of just deleting blobs\n --delete_all : (bool) if there are multiple matching files,\n allow the deletion of multiple files.\n Otherwise do not delete anything.\n --sd_hash= sd_hash : (str) delete by file sd hash\n --file_name file_name : (str) delete by file name in downloads folder\n --stream_hash= stream_hash : (str) delete by file stream hash\n --rowid= rowid : (int) delete by file row id\n --claim_id= claim_id : (str) delete by file claim id\n --txid= txid : (str) delete by file claim txid\n --nout= nout : (int) delete by file claim nout\n --claim_name= claim_name : (str) delete by file claim name\n --channel_claim_id= channel_claim_id : (str) delete by file channel claim id\n --channel_name= channel_name : (str) delete by file channel claim name\n\nReturns:\n (bool) true if deletion was successful",
"title": "file_delete"
},
{
"location": "/cli/#file_list",
"text": "List files limited by optional filters\n\nUsage:\n file_list [--sd_hash= sd_hash ] [--file_name= file_name ] [--stream_hash= stream_hash ]\n [--rowid= rowid ] [--claim_id= claim_id ] [--outpoint= outpoint ] [--txid= txid ] [--nout= nout ]\n [--channel_claim_id= channel_claim_id ] [--channel_name= channel_name ]\n [--claim_name= claim_name ] [--full_status] [--sort= sort_method ...]\n\n\nOptions:\n --sd_hash= sd_hash : (str) get file with matching sd hash\n --file_name= file_name : (str) get file with matching file name in the\n downloads folder\n --stream_hash= stream_hash : (str) get file with matching stream hash\n --rowid= rowid : (int) get file with matching row id\n --claim_id= claim_id : (str) get file with matching claim id\n --outpoint= outpoint : (str) get file with matching claim outpoint\n --txid= txid : (str) get file with matching claim txid\n --nout= nout : (int) get file with matching claim nout\n --channel_claim_id= channel_claim_id : (str) get file with matching channel claim id\n --channel_name= channel_name : (str) get file with matching channel name\n --claim_name= claim_name : (str) get file with matching claim name\n --full_status : (bool) full status, populate the\n 'message' and 'size' fields\n --sort= sort_method : (str) sort by any property, like 'file_name'\n or 'metadata.author'; to specify direction\n append ',asc' or ',desc'\n\nReturns:\n (list) List of files\n\n [\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) None if full_status is false or if claim is not found,\n 'outpoint': (str) None if full_status is false or if claim is not found,\n 'txid': (str) None if full_status is false or if claim is not found,\n 'nout': (int) None if full_status is false or if claim is not found,\n 'metadata': (dict) None if full_status is false or if claim is not found,\n 'channel_claim_id': (str) None if full_status is false or if claim is not found or signed,\n 'channel_name': (str) None if full_status is false or if claim is not found or signed,\n 'claim_name': (str) None if full_status is false or if claim is not found\n },\n ]",
"title": "file_list"
},
{
"location": "/cli/#file_reflect",
"text": "Reflect all the blobs in a file matching the filter criteria\n\nUsage:\n file_reflect [--sd_hash= sd_hash ] [--file_name= file_name ]\n [--stream_hash= stream_hash ] [--rowid= rowid ]\n [--reflector= reflector ]\n\n\nOptions:\n --sd_hash= sd_hash : (str) get file with matching sd hash\n --file_name= file_name : (str) get file with matching file name in the\n downloads folder\n --stream_hash= stream_hash : (str) get file with matching stream hash\n --rowid= rowid : (int) get file with matching row id\n --reflector= reflector : (str) reflector server, ip address or url\n by default choose a server from the config\n\nReturns:\n (list) list of blobs reflected",
"title": "file_reflect"
},
{
"location": "/cli/#file_set_status",
"text": "Start or stop downloading a file\n\nUsage:\n file_set_status ( status | --status= status ) [--sd_hash= sd_hash ]\n [--file_name= file_name ] [--stream_hash= stream_hash ] [--rowid= rowid ]\n\n\nOptions:\n --status= status : (str) one of start or stop \n --sd_hash= sd_hash : (str) set status of file with matching sd hash\n --file_name= file_name : (str) set status of file with matching file name in the\n downloads folder\n --stream_hash= stream_hash : (str) set status of file with matching stream hash\n --rowid= rowid : (int) set status of file with matching row id\n\nReturns:\n (str) Confirmation message",
"title": "file_set_status"
},
{
"location": "/cli/#get",
"text": "Download stream from a LBRY name.\n\nUsage:\n get uri [ file_name | --file_name= file_name ] [ timeout | --timeout= timeout ]\n\n\n\nOptions:\n --uri= uri : (str) uri of the content to download\n --file_name= file_name : (str) specified name for the downloaded file\n --timeout= timeout : (int) download timeout in number of seconds\n\nReturns:\n (dict) Dictionary containing information about the stream\n {\n 'completed': (bool) true if download is completed,\n 'file_name': (str) name of file,\n 'download_directory': (str) download directory,\n 'points_paid': (float) credit paid to download file,\n 'stopped': (bool) true if download is stopped,\n 'stream_hash': (str) stream hash of file,\n 'stream_name': (str) stream name ,\n 'suggested_file_name': (str) suggested file name,\n 'sd_hash': (str) sd hash of file,\n 'download_path': (str) download path of file,\n 'mime_type': (str) mime type of file,\n 'key': (str) key attached to file,\n 'total_bytes': (int) file size in bytes, None if full_status is false,\n 'written_bytes': (int) written size in bytes,\n 'blobs_completed': (int) num_completed, None if full_status is false,\n 'blobs_in_stream': (int) None if full_status is false,\n 'status': (str) downloader status, None if full_status is false,\n 'claim_id': (str) claim id,\n 'outpoint': (str) claim outpoint string,\n 'txid': (str) claim txid,\n 'nout': (int) claim nout,\n 'metadata': (dict) claim metadata,\n 'channel_claim_id': (str) None if claim is not signed\n 'channel_name': (str) None if claim is not signed\n 'claim_name': (str) claim name\n }",
"title": "get"
},
{
"location": "/cli/#help",
"text": "Return a useful message for an API command\n\nUsage:\n help [ command | --command= command ]\n\n\nOptions:\n --command= command : (str) command to retrieve documentation for\n\nReturns:\n (str) Help message",
"title": "help"
},
{
"location": "/cli/#peer_list",
"text": "Get peers for blob hash\n\nUsage:\n peer_list ( blob_hash | --blob_hash= blob_hash ) [ timeout | --timeout= timeout ]\n\n\nOptions:\n --blob_hash= blob_hash : (str) find available peers for this blob hash\n --timeout= timeout : (int) peer search timeout in seconds\n\nReturns:\n (list) List of contact dictionaries {'host': peer ip , 'port': peer port , 'node_id': peer node id }",
"title": "peer_list"
},
{
"location": "/cli/#peer_ping",
"text": "Send a kademlia ping to the specified peer. If address and port are provided the peer is directly pinged,\nif not provided the peer is located first.\n\nUsage:\n peer_ping ( node_id | --node_id= node_id ) [ address | --address= address ] [ port | --port= port ]\n\n\nOptions:\n --address= address : (str) ip address of the peer\n --port= port : (int) udp port of the peer\n\nReturns:\n (str) pong, or {'error': error message } if an error is encountered",
"title": "peer_ping"
},
{
"location": "/cli/#publish",
"text": "Make a new name claim and publish associated data to lbrynet,\nupdate over existing claim if user already has a claim for name.\n\nFields required in the final Metadata are:\n 'title'\n 'description'\n 'author'\n 'language'\n 'license'\n 'nsfw'\n\nMetadata can be set by either using the metadata argument or by setting individual arguments\nfee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\nor sources. Individual arguments will overwrite the fields specified in metadata argument.\n\nUsage:\n publish ( name | --name= name ) ( bid | --bid= bid ) [--metadata= metadata ]\n [--file_path= file_path ] [--fee= fee ] [--title= title ]\n [--description= description ] [--author= author ] [--language= language ]\n [--license= license ] [--license_url= license_url ] [--thumbnail= thumbnail ]\n [--preview= preview ] [--nsfw= nsfw ] [--sources= sources ]\n [--channel_name= channel_name ] [--channel_id= channel_id ]\n [--claim_address= claim_address ] [--change_address= change_address ]\n\n\nOptions:\n --name= name : (str) name of the content\n --bid= bid : (decimal) amount to back the claim\n --metadata= metadata : (dict) ClaimDict to associate with the claim.\n --file_path= file_path : (str) path to file to be associated with name. If provided,\n a lbry stream of this file will be used in 'sources'.\n If no path is given but a sources dict is provided,\n it will be used. If neither are provided, an\n error is raised.\n --fee= fee : (dict) Dictionary representing key fee to download content:\n {\n 'currency': currency_symbol,\n 'amount': decimal,\n 'address': str, optional\n }\n supported currencies: LBC, USD, BTC\n If an address is not provided a new one will be\n automatically generated. Default fee is zero.\n --title= title : (str) title of the publication\n --description= description : (str) description of the publication\n --author= author : (str) author of the publication\n --language= language : (str) language of the publication\n --license= license : (str) publication license\n --license_url= license_url : (str) publication license url\n --thumbnail= thumbnail : (str) thumbnail url\n --preview= preview : (str) preview url\n --nsfw= nsfw : (bool) whether the content is nsfw\n --sources= sources : (str) {'lbry_sd_hash': sd_hash} specifies sd hash of file\n --channel_name= channel_name : (str) name of the publisher channel name in the wallet\n --channel_id= channel_id : (str) claim id of the publisher channel, does not check\n for channel claim being in the wallet. This allows\n publishing to a channel where only the certificate\n private key is in the wallet.\n --claim_address= claim_address : (str) address where the claim is sent to, if not specified\n new address will automatically be created\n\nReturns:\n (dict) Dictionary containing result of the claim\n {\n 'tx' : (str) hex encoded transaction\n 'txid' : (str) txid of resulting claim\n 'nout' : (int) nout of the resulting claim\n 'fee' : (decimal) fee paid for the claim transaction\n 'claim_id' : (str) claim ID of the resulting claim\n }",
"title": "publish"
},
{
"location": "/cli/#report_bug",
"text": "Report a bug to slack\n\nUsage:\n report_bug ( message | --message= message )\n\n\nOptions:\n --message= message : (str) Description of the bug\n\nReturns:\n (bool) true if successful",
"title": "report_bug"
},
{
"location": "/cli/#resolve",
"text": "Resolve given LBRY URIs\n\nUsage:\n resolve [--force] ( uri | --uri= uri ) [ uris ...]\n\n\nOptions:\n --force : (bool) force refresh and ignore cache\n --uri= uri : (str) uri to resolve\n --uris= uris : (list) uris to resolve\n\nReturns:\n Dictionary of results, keyed by uri\n ' uri ': {\n If a resolution error occurs:\n 'error': Error message\n\n If the uri resolves to a channel or a claim in a channel:\n 'certificate': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the certificate claim,\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}],\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n\n If the uri resolves to a channel:\n 'claims_in_channel': (int) number of claims in the channel,\n\n If the uri resolves to a claim:\n 'claim': {\n 'address': (str) claim address,\n 'amount': (float) claim amount,\n 'effective_amount': (float) claim amount including supports,\n 'claim_id': (str) claim id,\n 'claim_sequence': (int) claim sequence number,\n 'decoded_claim': (bool) whether or not the claim value was decoded,\n 'height': (int) claim height,\n 'depth': (int) claim depth,\n 'has_signature': (bool) included if decoded_claim\n 'name': (str) claim name,\n 'permanent_url': (str) permanent url of the claim,\n 'channel_name': (str) channel name if claim is in a channel\n 'supports: (list) list of supports [{'txid': (str) txid,\n 'nout': (int) nout,\n 'amount': (float) amount}]\n 'txid': (str) claim txid,\n 'nout': (str) claim nout,\n 'signature_is_valid': (bool), included if has_signature,\n 'value': ClaimDict if decoded, otherwise hex string\n }\n }",
"title": "resolve"
},
{
"location": "/cli/#resolve_name",
"text": "Resolve stream info from a LBRY name\n\nUsage:\n resolve_name ( name | --name= name ) [--force]\n\n\nOptions:\n --name= name : (str) the name to resolve\n --force : (bool) force refresh and do not check cache\n\nReturns:\n (dict) Metadata dictionary from name claim, None if the name is not\n resolvable",
"title": "resolve_name"
},
{
"location": "/cli/#routing_table_get",
"text": "Get DHT routing information\n\nUsage:\n routing_table_get\n\n\nOptions:\n None\n\nReturns:\n (dict) dictionary containing routing and contact information\n {\n buckets : {\n bucket index : [\n {\n address : (str) peer address,\n port : (int) peer udp port\n node_id : (str) peer node id,\n blobs : (list) blob hashes announced by peer\n }\n ]\n },\n contacts : (list) contact node ids,\n blob_hashes : (list) all of the blob hashes stored by peers in the list of buckets,\n node_id : (str) the local dht node id\n }",
"title": "routing_table_get"
},
{
"location": "/cli/#settings_get",
"text": "Get daemon settings\n\nUsage:\n settings_get\n\n\nOptions:\n None\n\nReturns:\n (dict) Dictionary of daemon settings\n See ADJUSTABLE_SETTINGS in lbrynet/conf.py for full list of settings",
"title": "settings_get"
},
{
"location": "/cli/#settings_set",
"text": "Set daemon settings\n\nUsage:\n settings_set [--download_directory= download_directory ]\n [--data_rate= data_rate ]\n [--download_timeout= download_timeout ]\n [--peer_port= peer_port ]\n [--max_key_fee= max_key_fee ]\n [--disable_max_key_fee= disable_max_key_fee ]\n [--use_upnp= use_upnp ]\n [--run_reflector_server= run_reflector_server ]\n [--cache_time= cache_time ]\n [--reflect_uploads= reflect_uploads ]\n [--share_usage_data= share_usage_data ]\n [--peer_search_timeout= peer_search_timeout ]\n [--sd_download_timeout= sd_download_timeout ]\n [--auto_renew_claim_height_delta= auto_renew_claim_height_delta ]\n\n\nOptions:\n --download_directory= download_directory : (str) path of download directory\n --data_rate= data_rate : (float) 0.0001\n --download_timeout= download_timeout : (int) 180\n --peer_port= peer_port : (int) 3333\n --max_key_fee= max_key_fee : (dict) maximum key fee for downloads,\n in the format:\n {\n 'currency': currency_symbol ,\n 'amount': amount \n }.\n In the CLI, it must be an escaped JSON string\n Supported currency symbols: LBC, USD, BTC\n --disable_max_key_fee= disable_max_key_fee : (bool) False\n --use_upnp= use_upnp : (bool) True\n --run_reflector_server= run_reflector_server : (bool) False\n --cache_time= cache_time : (int) 150\n --reflect_uploads= reflect_uploads : (bool) True\n --share_usage_data= share_usage_data : (bool) True\n --peer_search_timeout= peer_search_timeout : (int) 3\n --sd_download_timeout= sd_download_timeout : (int) 3\n --auto_renew_claim_height_delta= auto_renew_claim_height_delta : (int) 0\n claims set to expire within this many blocks will be\n automatically renewed after startup (if set to 0, renews\n will not be made automatically)\n\nReturns:\n (dict) Updated dictionary of daemon settings",
"title": "settings_set"
},
{
"location": "/cli/#status",
"text": "Get daemon status\n\nUsage:\n status\n\n\nOptions:\n None\n\nReturns:\n (dict) lbrynet-daemon status\n {\n 'installation_id': (str) installation id - base58,\n 'is_running': (bool),\n 'is_first_run': bool,\n 'skipped_components': (list) [names of skipped components (str)],\n 'startup_status': { Does not include components which have been skipped\n 'database': (bool),\n 'wallet': (bool),\n 'session': (bool),\n 'dht': (bool),\n 'hash_announcer': (bool),\n 'stream_identifier': (bool),\n 'file_manager': (bool),\n 'blob_manager': (bool),\n 'blockchain_headers': (bool),\n 'peer_protocol_server': (bool),\n 'reflector': (bool),\n 'upnp': (bool),\n 'exchange_rate_manager': (bool),\n },\n 'connection_status': {\n 'code': (str) connection status code,\n 'message': (str) connection status message\n },\n 'blockchain_headers': {\n 'downloading_headers': (bool),\n 'download_progress': (float) 0-100.0\n },\n 'wallet': {\n 'blocks': (int) local blockchain height,\n 'blocks_behind': (int) remote_height - local_height,\n 'best_blockhash': (str) block hash of most recent block,\n 'is_encrypted': (bool),\n 'is_locked': (bool),\n },\n 'dht': {\n 'node_id': (str) lbry dht node id - hex encoded,\n 'peers_in_routing_table': (int) the number of peers in the routing table,\n },\n 'blob_manager': {\n 'finished_blobs': (int) number of finished blobs in the blob manager,\n },\n 'hash_announcer': {\n 'announce_queue_size': (int) number of blobs currently queued to be announced\n },\n 'file_manager': {\n 'managed_files': (int) count of files in the file manager,\n }\n }",
"title": "status"
},
{
"location": "/cli/#stream_availability",
"text": "Get stream availability for lbry uri\n\nUsage:\n stream_availability ( uri | --uri= uri )\n [ search_timeout | --search_timeout= search_timeout ]\n [ blob_timeout | --blob_timeout= blob_timeout ]\n\n\nOptions:\n --uri= uri : (str) check availability for this uri\n --search_timeout= search_timeout : (int) how long to search for peers for the blob\n in the dht\n --blob_timeout= blob_timeout : (int) how long to try downloading from a peer\n\nReturns:\n (dict) {\n 'is_available': bool ,\n 'did_decode': bool ,\n 'did_resolve': bool ,\n 'is_stream': bool ,\n 'num_blobs_in_stream': int ,\n 'sd_hash': str ,\n 'sd_blob_availability': dict see `blob_availability`,\n 'head_blob_hash': str ,\n 'head_blob_availability': dict see `blob_availability`,\n 'use_upnp': bool ,\n 'upnp_redirect_is_set': bool ,\n 'error': None | str error message\n }",
"title": "stream_availability"
},
{
"location": "/cli/#stream_cost_estimate",
"text": "Get estimated cost for a lbry stream\n\nUsage:\n stream_cost_estimate ( uri | --uri= uri ) [ size | --size= size ]\n\n\nOptions:\n --uri= uri : (str) uri to use\n --size= size : (float) stream size in bytes. if provided an sd blob won't be\n downloaded.\n\nReturns:\n (float) Estimated cost in lbry credits, returns None if uri is not\n resolvable",
"title": "stream_cost_estimate"
},
{
"location": "/cli/#transaction_list",
"text": "List transactions belonging to wallet\n\nUsage:\n transaction_list\n\n\nOptions:\n None\n\nReturns:\n (list) List of transactions\n\n {\n claim_info : (list) claim info if in txn [{\n address : (str) address of claim,\n balance_delta : (float) bid amount,\n amount : (float) claim amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n nout : (int) nout\n }],\n abandon_info : (list) abandon info if in txn [{\n address : (str) address of abandoned claim,\n balance_delta : (float) returned amount,\n amount : (float) claim amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n nout : (int) nout\n }],\n confirmations : (int) number of confirmations for the txn,\n date : (str) date and time of txn,\n fee : (float) txn fee,\n support_info : (list) support info if in txn [{\n address : (str) address of support,\n balance_delta : (float) support amount,\n amount : (float) support amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n is_tip : (bool),\n nout : (int) nout\n }],\n timestamp : (int) timestamp,\n txid : (str) txn id,\n update_info : (list) update info if in txn [{\n address : (str) address of claim,\n balance_delta : (float) credited/debited\n amount : (float) absolute amount,\n claim_id : (str) claim id,\n claim_name : (str) claim name,\n nout : (int) nout\n }],\n value : (float) value of txn\n }",
"title": "transaction_list"
},
{
"location": "/cli/#transaction_show",
"text": "Get a decoded transaction from a txid\n\nUsage:\n transaction_show ( txid | --txid= txid )\n\n\nOptions:\n --txid= txid : (str) txid of the transaction\n\nReturns:\n (dict) JSON formatted transaction",
"title": "transaction_show"
},
{
"location": "/cli/#utxo_list",
"text": "List unspent transaction outputs\n\nUsage:\n utxo_list\n\n\nOptions:\n None\n\nReturns:\n (list) List of unspent transaction outputs (UTXOs)\n [\n {\n address : (str) the output address\n amount : (float) unspent amount\n height : (int) block height\n is_claim : (bool) is the tx a claim\n is_coinbase : (bool) is the tx a coinbase tx\n is_support : (bool) is the tx a support\n is_update : (bool) is the tx an update\n nout : (int) nout of the output\n txid : (str) txid of the output\n },\n ...\n ]",
"title": "utxo_list"
},
{
"location": "/cli/#version",
"text": "Get lbry version information\n\nUsage:\n version\n\n\nOptions:\n None\n\nReturns:\n (dict) Dictionary of lbry version information\n {\n 'build': (str) build type (e.g. dev , rc , release ),\n 'ip': (str) remote ip, if available,\n 'lbrynet_version': (str) lbrynet_version,\n 'lbryum_version': (str) lbryum_version,\n 'lbryschema_version': (str) lbryschema_version,\n 'os_release': (str) os release string\n 'os_system': (str) os name\n 'platform': (str) platform string\n 'processor': (str) processor type,\n 'python_version': (str) python version,\n }",
"title": "version"
},
{
"location": "/cli/#wallet_balance",
"text": "Return the balance of the wallet\n\nUsage:\n wallet_balance [ address | --address= address ] [--include_unconfirmed]\n\n\nOptions:\n --address= address : (str) If provided only the balance for this\n address will be given\n --include_unconfirmed : (bool) Include unconfirmed\n\nReturns:\n (float) amount of lbry credits in wallet",
"title": "wallet_balance"
},
{
"location": "/cli/#wallet_decrypt",
"text": "Decrypt an encrypted wallet, this will remove the wallet password\n\nUsage:\n wallet_decrypt\n\n\nOptions:\n None\n\nReturns:\n (bool) true if wallet is decrypted, otherwise false",
"title": "wallet_decrypt"
},
{
"location": "/cli/#wallet_encrypt",
"text": "Encrypt a wallet with a password, if the wallet is already encrypted this will update\nthe password\n\nUsage:\n wallet_encrypt ( new_password | --new_password= new_password )\n\n\nOptions:\n --new_password= new_password : (str) password string to be used for encrypting wallet\n\nReturns:\n (bool) true if wallet is decrypted, otherwise false",
"title": "wallet_encrypt"
},
{
"location": "/cli/#wallet_is_address_mine",
"text": "Checks if an address is associated with the current wallet.\n\nUsage:\n wallet_is_address_mine ( address | --address= address )\n\n\nOptions:\n --address= address : (str) address to check\n\nReturns:\n (bool) true, if address is associated with current wallet",
"title": "wallet_is_address_mine"
},
{
"location": "/cli/#wallet_list",
"text": "List wallet addresses\n\nUsage:\n wallet_list\n\n\nOptions:\n None\n\nReturns:\n List of wallet addresses",
"title": "wallet_list"
},
{
"location": "/cli/#wallet_new_address",
"text": "Generate a new wallet address\n\nUsage:\n wallet_new_address\n\n\nOptions:\n None\n\nReturns:\n (str) New wallet address in base58",
"title": "wallet_new_address"
},
{
"location": "/cli/#wallet_prefill_addresses",
"text": "Create new addresses, each containing `amount` credits\n\nUsage:\n wallet_prefill_addresses [--no_broadcast]\n ( num_addresses | --num_addresses= num_addresses )\n ( amount | --amount= amount )\n\n\nOptions:\n --no_broadcast : (bool) whether to broadcast or not\n --num_addresses= num_addresses : (int) num of addresses to create\n --amount= amount : (float) initial amount in each address\n\nReturns:\n (dict) the resulting transaction",
"title": "wallet_prefill_addresses"
},
{
"location": "/cli/#wallet_public_key",
"text": "Get public key from wallet address\n\nUsage:\n wallet_public_key ( address | --address= address )\n\n\nOptions:\n --address= address : (str) address for which to get the public key\n\nReturns:\n (list) list of public keys associated with address.\n Could contain more than one public key if multisig.",
"title": "wallet_public_key"
},
{
"location": "/cli/#wallet_send",
"text": "Send credits. If given an address, send credits to it. If given a claim id, send a tip\nto the owner of a claim specified by uri. A tip is a claim support where the recipient\nof the support is the claim address for the claim being supported.\n\nUsage:\n wallet_send ( amount | --amount= amount )\n (( address | --address= address ) | ( claim_id | --claim_id= claim_id ))\n\n\nOptions:\n --amount= amount : (Decimal) amount of credit to send\n --address= address : (str) address to send credits to\n --claim_id= claim_id : (str) claim_id of the claim to send to tip to\n\nReturns:\n If sending to an address:\n (bool) true if payment successfully scheduled\n\n If sending a claim tip:\n (dict) Dictionary containing the result of the support\n {\n txid : (str) txid of resulting support claim\n nout : (int) nout of the resulting support claim\n fee : (float) fee paid for the transaction\n }",
"title": "wallet_send"
},
{
"location": "/cli/#wallet_unlock",
"text": "Unlock an encrypted wallet\n\nUsage:\n wallet_unlock ( password | --password= password )\n\n\nOptions:\n --password= password : (str) password for unlocking wallet\n\nReturns:\n (bool) true if wallet is unlocked, otherwise false",
"title": "wallet_unlock"
},
{
"location": "/cli/#wallet_unused_address",
"text": "Return an address containing no balance, will create\na new address if there is none.\n\nUsage:\n wallet_unused_address\n\n\nOptions:\n None\n\nReturns:\n (str) Unused wallet address in base58",
"title": "wallet_unused_address"
}
]
}
+7 -5
View File
@@ -4,20 +4,22 @@
share_usage_data: True
lbryum_servers:
- lbryumx1.lbry.io:50001
- lbryumx2.lbry.io:50001
- lbryumx1.lbry.com:50001
- lbryumx2.lbry.com:50001
- lbryumx4.lbry.com:50001
blockchain_name: lbrycrd_main
data_dir: /home/lbry/.lbrynet
download_directory: /home/lbry/downloads
delete_blobs_on_remove: True
save_blobs: true
save_files: false
dht_node_port: 4444
peer_port: 3333
use_upnp: True
use_upnp: true
#components_to_skip:
# - peer_protocol_server
# - hash_announcer
# - blob_server
# - dht
+8
View File
@@ -0,0 +1,8 @@
__version__ = "1.0.0"
from lbry.wallet import Account, Wallet, WalletManager
from lbry.blockchain import Ledger, RegTestLedger, TestNetLedger
from lbry.blockchain import Transaction, Output, Input
from lbry.blockchain import dewies_to_lbc, lbc_to_dewies, dict_values_to_lbc
from lbry.service import API, Daemon, FullNode, LightClient
from lbry.db.database import Database
from lbry.conf import Config
+6
View File
@@ -0,0 +1,6 @@
from lbry.utils import get_lbry_hash_obj
MAX_BLOB_SIZE = 2 * 2 ** 20
# digest_size is in bytes, and blob hashes are hex encoded
BLOBHASH_LENGTH = get_lbry_hash_obj().digest_size * 2
+340
View File
@@ -0,0 +1,340 @@
import os
import re
import asyncio
import binascii
import logging
import typing
import contextlib
from io import BytesIO
from cryptography.hazmat.primitives.ciphers import Cipher, modes
from cryptography.hazmat.primitives.ciphers.algorithms import AES
from cryptography.hazmat.primitives.padding import PKCS7
from cryptography.hazmat.backends import default_backend
from lbry.utils import get_lbry_hash_obj
from lbry.error import DownloadCancelledError, InvalidBlobHashError, InvalidDataError
from lbry.blob import MAX_BLOB_SIZE, BLOBHASH_LENGTH
from lbry.blob.blob_info import BlobInfo
from lbry.blob.writer import HashBlobWriter
log = logging.getLogger(__name__)
HEXMATCH = re.compile("^[a-f,0-9]+$")
BACKEND = default_backend()
def is_valid_blobhash(blobhash: str) -> bool:
"""Checks whether the blobhash is the correct length and contains only
valid characters (0-9, a-f)
@param blobhash: string, the blobhash to check
@return: True/False
"""
return len(blobhash) == BLOBHASH_LENGTH and HEXMATCH.match(blobhash)
def encrypt_blob_bytes(key: bytes, iv: bytes, unencrypted: bytes) -> typing.Tuple[bytes, str]:
cipher = Cipher(AES(key), modes.CBC(iv), backend=BACKEND)
padder = PKCS7(AES.block_size).padder()
encryptor = cipher.encryptor()
encrypted = encryptor.update(padder.update(unencrypted) + padder.finalize()) + encryptor.finalize()
digest = get_lbry_hash_obj()
digest.update(encrypted)
return encrypted, digest.hexdigest()
def decrypt_blob_bytes(data: bytes, length: int, key: bytes, iv: bytes) -> bytes:
if len(data) != length:
raise ValueError("unexpected length")
cipher = Cipher(AES(key), modes.CBC(iv), backend=BACKEND)
unpadder = PKCS7(AES.block_size).unpadder()
decryptor = cipher.decryptor()
return unpadder.update(decryptor.update(data) + decryptor.finalize()) + unpadder.finalize()
class AbstractBlob:
"""
A chunk of data (up to 2MB) available on the network which is specified by a sha384 hash
This class is non-io specific
"""
__slots__ = [
'loop',
'blob_hash',
'length',
'blob_completed_callback',
'blob_directory',
'writers',
'verified',
'writing',
'readers'
]
def __init__(self, loop: asyncio.AbstractEventLoop, blob_hash: str, length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], asyncio.Task]] = None,
blob_directory: typing.Optional[str] = None):
self.loop = loop
self.blob_hash = blob_hash
self.length = length
self.blob_completed_callback = blob_completed_callback
self.blob_directory = blob_directory
self.writers: typing.Dict[typing.Tuple[typing.Optional[str], typing.Optional[int]], HashBlobWriter] = {}
self.verified: asyncio.Event = asyncio.Event(loop=self.loop)
self.writing: asyncio.Event = asyncio.Event(loop=self.loop)
self.readers: typing.List[typing.BinaryIO] = []
if not is_valid_blobhash(blob_hash):
raise InvalidBlobHashError(blob_hash)
def __del__(self):
if self.writers or self.readers:
log.warning("%s not closed before being garbage collected", self.blob_hash)
self.close()
@contextlib.contextmanager
def _reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
raise NotImplementedError()
@contextlib.contextmanager
def reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
if not self.is_readable():
raise OSError(f"{str(type(self))} not readable, {len(self.readers)} readers {len(self.writers)} writers")
with self._reader_context() as reader:
try:
self.readers.append(reader)
yield reader
finally:
if reader in self.readers:
self.readers.remove(reader)
def _write_blob(self, blob_bytes: bytes):
raise NotImplementedError()
def set_length(self, length) -> None:
if self.length is not None and length == self.length:
return
if self.length is None and 0 <= length <= MAX_BLOB_SIZE:
self.length = length
return
log.warning("Got an invalid length. Previous length: %s, Invalid length: %s", self.length, length)
def get_length(self) -> typing.Optional[int]:
return self.length
def get_is_verified(self) -> bool:
return self.verified.is_set()
def is_readable(self) -> bool:
return self.verified.is_set()
def is_writeable(self) -> bool:
return not self.writing.is_set()
def write_blob(self, blob_bytes: bytes):
if not self.is_writeable():
raise OSError("cannot open blob for writing")
try:
self.writing.set()
self._write_blob(blob_bytes)
finally:
self.writing.clear()
def close(self):
while self.writers:
_, writer = self.writers.popitem()
if writer and writer.finished and not writer.finished.done() and not self.loop.is_closed():
writer.finished.cancel()
while self.readers:
reader = self.readers.pop()
if reader:
reader.close()
def delete(self):
self.close()
self.verified.clear()
self.length = None
async def sendfile(self, writer: asyncio.StreamWriter) -> int:
"""
Read and send the file to the writer and return the number of bytes sent
"""
if not self.is_readable():
raise OSError('blob files cannot be read')
with self.reader_context() as handle:
try:
return await self.loop.sendfile(writer.transport, handle, count=self.get_length())
except (ConnectionError, BrokenPipeError, RuntimeError, OSError, AttributeError):
return -1
def decrypt(self, key: bytes, iv: bytes) -> bytes:
"""
Decrypt a BlobFile to plaintext bytes
"""
with self.reader_context() as reader:
return decrypt_blob_bytes(reader.read(), self.length, key, iv)
@classmethod
async def create_from_unencrypted(
cls, loop: asyncio.AbstractEventLoop, blob_dir: typing.Optional[str], key: bytes, iv: bytes,
unencrypted: bytes, blob_num: int,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], None]] = None) -> BlobInfo:
"""
Create an encrypted BlobFile from plaintext bytes
"""
blob_bytes, blob_hash = encrypt_blob_bytes(key, iv, unencrypted)
length = len(blob_bytes)
blob = cls(loop, blob_hash, length, blob_completed_callback, blob_dir)
writer = blob.get_blob_writer()
writer.write(blob_bytes)
await blob.verified.wait()
return BlobInfo(blob_num, length, binascii.hexlify(iv).decode(), blob_hash)
def save_verified_blob(self, verified_bytes: bytes):
if self.verified.is_set():
return
if self.is_writeable():
self._write_blob(verified_bytes)
self.verified.set()
if self.blob_completed_callback:
self.blob_completed_callback(self)
def get_blob_writer(self, peer_address: typing.Optional[str] = None,
peer_port: typing.Optional[int] = None) -> HashBlobWriter:
if (peer_address, peer_port) in self.writers and not self.writers[(peer_address, peer_port)].closed():
raise OSError(f"attempted to download blob twice from {peer_address}:{peer_port}")
fut = asyncio.Future(loop=self.loop)
writer = HashBlobWriter(self.blob_hash, self.get_length, fut)
self.writers[(peer_address, peer_port)] = writer
def remove_writer(_):
if (peer_address, peer_port) in self.writers:
del self.writers[(peer_address, peer_port)]
fut.add_done_callback(remove_writer)
def writer_finished_callback(finished: asyncio.Future):
try:
err = finished.exception()
if err:
raise err
verified_bytes = finished.result()
while self.writers:
_, other = self.writers.popitem()
if other is not writer:
other.close_handle()
self.save_verified_blob(verified_bytes)
except (InvalidBlobHashError, InvalidDataError) as error:
log.warning("writer error downloading %s: %s", self.blob_hash[:8], str(error))
except (DownloadCancelledError, asyncio.CancelledError, asyncio.TimeoutError):
pass
fut.add_done_callback(writer_finished_callback)
return writer
class BlobBuffer(AbstractBlob):
"""
An in-memory only blob
"""
def __init__(self, loop: asyncio.AbstractEventLoop, blob_hash: str, length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], asyncio.Task]] = None,
blob_directory: typing.Optional[str] = None):
self._verified_bytes: typing.Optional[BytesIO] = None
super().__init__(loop, blob_hash, length, blob_completed_callback, blob_directory)
@contextlib.contextmanager
def _reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
if not self.is_readable():
raise OSError("cannot open blob for reading")
try:
yield self._verified_bytes
finally:
if self._verified_bytes:
self._verified_bytes.close()
self._verified_bytes = None
self.verified.clear()
def _write_blob(self, blob_bytes: bytes):
if self._verified_bytes:
raise OSError("already have bytes for blob")
self._verified_bytes = BytesIO(blob_bytes)
def delete(self):
if self._verified_bytes:
self._verified_bytes.close()
self._verified_bytes = None
return super().delete()
def __del__(self):
super().__del__()
if self._verified_bytes:
self.delete()
class BlobFile(AbstractBlob):
"""
A blob existing on the local file system
"""
def __init__(self, loop: asyncio.AbstractEventLoop, blob_hash: str, length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], asyncio.Task]] = None,
blob_directory: typing.Optional[str] = None):
super().__init__(loop, blob_hash, length, blob_completed_callback, blob_directory)
if not blob_directory or not os.path.isdir(blob_directory):
raise OSError(f"invalid blob directory '{blob_directory}'")
self.file_path = os.path.join(self.blob_directory, self.blob_hash)
if self.file_exists:
file_size = int(os.stat(self.file_path).st_size)
if length and length != file_size:
log.warning("expected %s to be %s bytes, file has %s", self.blob_hash, length, file_size)
self.delete()
else:
self.length = file_size
self.verified.set()
@property
def file_exists(self):
return os.path.isfile(self.file_path)
def is_writeable(self) -> bool:
return super().is_writeable() and not os.path.isfile(self.file_path)
def get_blob_writer(self, peer_address: typing.Optional[str] = None,
peer_port: typing.Optional[str] = None) -> HashBlobWriter:
if self.file_exists:
raise OSError(f"File already exists '{self.file_path}'")
return super().get_blob_writer(peer_address, peer_port)
@contextlib.contextmanager
def _reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
handle = open(self.file_path, 'rb')
try:
yield handle
finally:
handle.close()
def _write_blob(self, blob_bytes: bytes):
with open(self.file_path, 'wb') as f:
f.write(blob_bytes)
def delete(self):
if os.path.isfile(self.file_path):
os.remove(self.file_path)
return super().delete()
@classmethod
async def create_from_unencrypted(
cls, loop: asyncio.AbstractEventLoop, blob_dir: typing.Optional[str], key: bytes, iv: bytes,
unencrypted: bytes, blob_num: int,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'],
asyncio.Task]] = None) -> BlobInfo:
if not blob_dir or not os.path.isdir(blob_dir):
raise OSError(f"cannot create blob in directory: '{blob_dir}'")
return await super().create_from_unencrypted(
loop, blob_dir, key, iv, unencrypted, blob_num, blob_completed_callback
)
+26
View File
@@ -0,0 +1,26 @@
import typing
class BlobInfo:
__slots__ = [
'blob_hash',
'blob_num',
'length',
'iv',
]
def __init__(self, blob_num: int, length: int, iv: str, blob_hash: typing.Optional[str] = None):
self.blob_hash = blob_hash
self.blob_num = blob_num
self.length = length
self.iv = iv
def as_dict(self) -> typing.Dict:
d = {
'length': self.length,
'blob_num': self.blob_num,
'iv': self.iv,
}
if self.blob_hash: # non-terminator blobs have a blob hash
d['blob_hash'] = self.blob_hash
return d
+137
View File
@@ -0,0 +1,137 @@
import os
import typing
import asyncio
import logging
from lbry.utils import LRUCache
from lbry.blob.blob_file import is_valid_blobhash, BlobFile, BlobBuffer, AbstractBlob
from lbry.stream.descriptor import StreamDescriptor
from lbry.connection_manager import ConnectionManager
if typing.TYPE_CHECKING:
from lbry.conf import Config
from lbry.dht.protocol.data_store import DictDataStore
class SQLiteStorage:
pass
log = logging.getLogger(__name__)
class BlobManager:
def __init__(self, loop: asyncio.AbstractEventLoop, blob_dir: str, storage: 'SQLiteStorage', config: 'Config',
node_data_store: typing.Optional['DictDataStore'] = None):
"""
This class stores blobs on the hard disk
blob_dir - directory where blobs are stored
storage - SQLiteStorage object
"""
self.loop = loop
self.blob_dir = blob_dir
self.storage = storage
self._node_data_store = node_data_store
self.completed_blob_hashes: typing.Set[str] = set() if not self._node_data_store\
else self._node_data_store.completed_blobs
self.blobs: typing.Dict[str, AbstractBlob] = {}
self.config = config
self.decrypted_blob_lru_cache = None if not self.config.blob_lru_cache_size else LRUCache(
self.config.blob_lru_cache_size)
self.connection_manager = ConnectionManager(loop)
def _get_blob(self, blob_hash: str, length: typing.Optional[int] = None):
if self.config.save_blobs or (
is_valid_blobhash(blob_hash) and os.path.isfile(os.path.join(self.blob_dir, blob_hash))):
return BlobFile(
self.loop, blob_hash, length, self.blob_completed, self.blob_dir
)
return BlobBuffer(
self.loop, blob_hash, length, self.blob_completed, self.blob_dir
)
def get_blob(self, blob_hash, length: typing.Optional[int] = None):
if blob_hash in self.blobs:
if self.config.save_blobs and isinstance(self.blobs[blob_hash], BlobBuffer):
buffer = self.blobs.pop(blob_hash)
if blob_hash in self.completed_blob_hashes:
self.completed_blob_hashes.remove(blob_hash)
self.blobs[blob_hash] = self._get_blob(blob_hash, length)
if buffer.is_readable():
with buffer.reader_context() as reader:
self.blobs[blob_hash].write_blob(reader.read())
if length and self.blobs[blob_hash].length is None:
self.blobs[blob_hash].set_length(length)
else:
self.blobs[blob_hash] = self._get_blob(blob_hash, length)
return self.blobs[blob_hash]
def is_blob_verified(self, blob_hash: str, length: typing.Optional[int] = None) -> bool:
if not is_valid_blobhash(blob_hash):
raise ValueError(blob_hash)
if not os.path.isfile(os.path.join(self.blob_dir, blob_hash)):
return False
if blob_hash in self.blobs:
return self.blobs[blob_hash].get_is_verified()
return self._get_blob(blob_hash, length).get_is_verified()
async def setup(self) -> bool:
def get_files_in_blob_dir() -> typing.Set[str]:
if not self.blob_dir:
return set()
return {
item.name for item in os.scandir(self.blob_dir) if is_valid_blobhash(item.name)
}
in_blobfiles_dir = await self.loop.run_in_executor(None, get_files_in_blob_dir)
to_add = await self.storage.sync_missing_blobs(in_blobfiles_dir)
if to_add:
self.completed_blob_hashes.update(to_add)
if self.config.track_bandwidth:
self.connection_manager.start()
return True
def stop(self):
self.connection_manager.stop()
while self.blobs:
_, blob = self.blobs.popitem()
blob.close()
self.completed_blob_hashes.clear()
def get_stream_descriptor(self, sd_hash):
return StreamDescriptor.from_stream_descriptor_blob(self.loop, self.blob_dir, self.get_blob(sd_hash))
def blob_completed(self, blob: AbstractBlob) -> asyncio.Task:
if blob.blob_hash is None:
raise Exception("Blob hash is None")
if not blob.length:
raise Exception("Blob has a length of 0")
if isinstance(blob, BlobFile):
if blob.blob_hash not in self.completed_blob_hashes:
self.completed_blob_hashes.add(blob.blob_hash)
return self.loop.create_task(self.storage.add_blobs((blob.blob_hash, blob.length), finished=True))
else:
return self.loop.create_task(self.storage.add_blobs((blob.blob_hash, blob.length), finished=False))
def check_completed_blobs(self, blob_hashes: typing.List[str]) -> typing.List[str]:
"""Returns of the blobhashes_to_check, which are valid"""
return [blob_hash for blob_hash in blob_hashes if self.is_blob_verified(blob_hash)]
def delete_blob(self, blob_hash: str):
if not is_valid_blobhash(blob_hash):
raise Exception("invalid blob hash to delete")
if blob_hash not in self.blobs:
if self.blob_dir and os.path.isfile(os.path.join(self.blob_dir, blob_hash)):
os.remove(os.path.join(self.blob_dir, blob_hash))
else:
self.blobs.pop(blob_hash).delete()
if blob_hash in self.completed_blob_hashes:
self.completed_blob_hashes.remove(blob_hash)
async def delete_blobs(self, blob_hashes: typing.List[str], delete_from_db: typing.Optional[bool] = True):
for blob_hash in blob_hashes:
self.delete_blob(blob_hash)
if delete_from_db:
await self.storage.delete_blobs_from_db(blob_hashes)
+68
View File
@@ -0,0 +1,68 @@
import typing
import logging
import asyncio
from io import BytesIO
from lbry.error import InvalidBlobHashError, InvalidDataError
from lbry.utils import get_lbry_hash_obj
log = logging.getLogger(__name__)
class HashBlobWriter:
def __init__(self, expected_blob_hash: str, get_length: typing.Callable[[], int],
finished: asyncio.Future):
self.expected_blob_hash = expected_blob_hash
self.get_length = get_length
self.buffer = BytesIO()
self.finished = finished
self.finished.add_done_callback(lambda *_: self.close_handle())
self._hashsum = get_lbry_hash_obj()
self.len_so_far = 0
def __del__(self):
if self.buffer is not None:
log.warning("Garbage collection was called, but writer was not closed yet")
self.close_handle()
def calculate_blob_hash(self) -> str:
return self._hashsum.hexdigest()
def closed(self):
return self.buffer is None or self.buffer.closed
def write(self, data: bytes):
expected_length = self.get_length()
if not expected_length:
raise OSError("unknown blob length")
if self.buffer is None:
log.warning("writer has already been closed")
if not self.finished.done():
self.finished.cancel()
return
raise OSError('I/O operation on closed file')
self._hashsum.update(data)
self.len_so_far += len(data)
if self.len_so_far > expected_length:
self.finished.set_exception(InvalidDataError(
f'Length so far is greater than the expected length. {self.len_so_far} to {expected_length}'
))
self.close_handle()
return
self.buffer.write(data)
if self.len_so_far == expected_length:
blob_hash = self.calculate_blob_hash()
if blob_hash != self.expected_blob_hash:
self.finished.set_exception(InvalidBlobHashError(
f"blob hash is {blob_hash} vs expected {self.expected_blob_hash}"
))
elif self.finished and not (self.finished.done() or self.finished.cancelled()):
self.finished.set_result(self.buffer.getvalue())
self.close_handle()
def close_handle(self):
if not self.finished.done():
self.finished.cancel()
if self.buffer is not None:
self.buffer.close()
self.buffer = None
File renamed without changes.
+253
View File
@@ -0,0 +1,253 @@
import asyncio
import time
import logging
import typing
import binascii
from typing import Optional
from lbry.error import InvalidBlobHashError, InvalidDataError
from lbry.blob_exchange.serialization import BlobResponse, BlobRequest
from lbry.utils import cache_concurrent
if typing.TYPE_CHECKING:
from lbry.blob.blob_file import AbstractBlob
from lbry.blob.writer import HashBlobWriter
from lbry.connection_manager import ConnectionManager
log = logging.getLogger(__name__)
class BlobExchangeClientProtocol(asyncio.Protocol):
def __init__(self, loop: asyncio.AbstractEventLoop, peer_timeout: typing.Optional[float] = 10,
connection_manager: typing.Optional['ConnectionManager'] = None):
self.loop = loop
self.peer_port: typing.Optional[int] = None
self.peer_address: typing.Optional[str] = None
self.transport: typing.Optional[asyncio.Transport] = None
self.peer_timeout = peer_timeout
self.connection_manager = connection_manager
self.writer: typing.Optional['HashBlobWriter'] = None
self.blob: typing.Optional['AbstractBlob'] = None
self._blob_bytes_received = 0
self._response_fut: typing.Optional[asyncio.Future] = None
self.buf = b''
# this is here to handle the race when the downloader is closed right as response_fut gets a result
self.closed = asyncio.Event(loop=self.loop)
def data_received(self, data: bytes):
if self.connection_manager:
if not self.peer_address:
addr_info = self.transport.get_extra_info('peername')
self.peer_address, self.peer_port = addr_info
# assert self.peer_address is not None
self.connection_manager.received_data(f"{self.peer_address}:{self.peer_port}", len(data))
if not self.transport or self.transport.is_closing():
log.warning("transport closing, but got more bytes from %s:%i\n%s", self.peer_address, self.peer_port,
binascii.hexlify(data))
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
return
if not self._response_fut:
log.warning("Protocol received data before expected, probable race on keep alive. Closing transport.")
return self.close()
if self._blob_bytes_received and not self.writer.closed():
return self._write(data)
response = BlobResponse.deserialize(self.buf + data)
if not response.responses and not self._response_fut.done():
self.buf += data
return
else:
self.buf = b''
if response.responses and self.blob:
blob_response = response.get_blob_response()
if blob_response and not blob_response.error and blob_response.blob_hash == self.blob.blob_hash:
# set the expected length for the incoming blob if we didn't know it
self.blob.set_length(blob_response.length)
elif blob_response and not blob_response.error and self.blob.blob_hash != blob_response.blob_hash:
# the server started sending a blob we didn't request
log.warning("%s started sending blob we didn't request %s instead of %s", self.peer_address,
blob_response.blob_hash, self.blob.blob_hash)
return
if response.responses:
log.debug("got response from %s:%i <- %s", self.peer_address, self.peer_port, response.to_dict())
# fire the Future with the response to our request
self._response_fut.set_result(response)
if response.blob_data and self.writer and not self.writer.closed():
# log.debug("got %i blob bytes from %s:%i", len(response.blob_data), self.peer_address, self.peer_port)
# write blob bytes if we're writing a blob and have blob bytes to write
self._write(response.blob_data)
def _write(self, data: bytes):
if len(data) > (self.blob.get_length() - self._blob_bytes_received):
data = data[:(self.blob.get_length() - self._blob_bytes_received)]
log.warning("got more than asked from %s:%d, probable sendfile bug", self.peer_address, self.peer_port)
self._blob_bytes_received += len(data)
try:
self.writer.write(data)
except OSError as err:
log.error("error downloading blob from %s:%i: %s", self.peer_address, self.peer_port, err)
if self._response_fut and not self._response_fut.done():
self._response_fut.set_exception(err)
except asyncio.TimeoutError as err:
log.error("%s downloading blob from %s:%i", str(err), self.peer_address, self.peer_port)
if self._response_fut and not self._response_fut.done():
self._response_fut.set_exception(err)
async def _download_blob(self) -> typing.Tuple[int, Optional['BlobExchangeClientProtocol']]: # pylint: disable=too-many-return-statements
"""
:return: download success (bool), connected protocol (BlobExchangeClientProtocol)
"""
start_time = time.perf_counter()
request = BlobRequest.make_request_for_blob_hash(self.blob.blob_hash)
blob_hash = self.blob.blob_hash
if not self.peer_address:
addr_info = self.transport.get_extra_info('peername')
self.peer_address, self.peer_port = addr_info
try:
msg = request.serialize()
log.debug("send request to %s:%i -> %s", self.peer_address, self.peer_port, msg.decode())
self.transport.write(msg)
if self.connection_manager:
self.connection_manager.sent_data(f"{self.peer_address}:{self.peer_port}", len(msg))
response: BlobResponse = await asyncio.wait_for(self._response_fut, self.peer_timeout, loop=self.loop)
availability_response = response.get_availability_response()
price_response = response.get_price_response()
blob_response = response.get_blob_response()
if self.closed.is_set():
msg = f"cancelled blob request for {blob_hash} immediately after we got a response"
log.warning(msg)
raise asyncio.CancelledError(msg)
if (not blob_response or blob_response.error) and\
(not availability_response or not availability_response.available_blobs):
log.warning("%s not in availability response from %s:%i", self.blob.blob_hash, self.peer_address,
self.peer_port)
log.warning(response.to_dict())
return self._blob_bytes_received, self.close()
elif availability_response and availability_response.available_blobs and \
availability_response.available_blobs != [self.blob.blob_hash]:
log.warning("blob availability response doesn't match our request from %s:%i",
self.peer_address, self.peer_port)
return self._blob_bytes_received, self.close()
elif not availability_response:
log.warning("response from %s:%i did not include an availability response (we requested %s)",
self.peer_address, self.peer_port, blob_hash)
return self._blob_bytes_received, self.close()
if not price_response or price_response.blob_data_payment_rate != 'RATE_ACCEPTED':
log.warning("data rate rejected by %s:%i", self.peer_address, self.peer_port)
return self._blob_bytes_received, self.close()
if not blob_response or blob_response.error:
log.warning("blob can't be downloaded from %s:%i", self.peer_address, self.peer_port)
return self._blob_bytes_received, self.close()
if not blob_response.error and blob_response.blob_hash != self.blob.blob_hash:
log.warning("incoming blob hash mismatch from %s:%i", self.peer_address, self.peer_port)
return self._blob_bytes_received, self.close()
if self.blob.length is not None and self.blob.length != blob_response.length:
log.warning("incoming blob unexpected length from %s:%i", self.peer_address, self.peer_port)
return self._blob_bytes_received, self.close()
msg = f"downloading {self.blob.blob_hash[:8]} from {self.peer_address}:{self.peer_port}," \
f" timeout in {self.peer_timeout}"
log.debug(msg)
msg = f"downloaded {self.blob.blob_hash[:8]} from {self.peer_address}:{self.peer_port}"
await asyncio.wait_for(self.writer.finished, self.peer_timeout, loop=self.loop)
log.info("%s at %fMB/s", msg,
round((float(self._blob_bytes_received) /
float(time.perf_counter() - start_time)) / 1000000.0, 2))
# await self.blob.finished_writing.wait() not necessary, but a dangerous change. TODO: is it needed?
return self._blob_bytes_received, self
except asyncio.TimeoutError:
return self._blob_bytes_received, self.close()
except (InvalidBlobHashError, InvalidDataError):
log.warning("invalid blob from %s:%i", self.peer_address, self.peer_port)
return self._blob_bytes_received, self.close()
def close(self):
self.closed.set()
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
if self.writer and not self.writer.closed():
self.writer.close_handle()
self._response_fut = None
self.writer = None
self.blob = None
if self.transport:
self.transport.close()
self.transport = None
self.buf = b''
async def download_blob(self, blob: 'AbstractBlob') -> typing.Tuple[int, Optional['BlobExchangeClientProtocol']]:
self.closed.clear()
blob_hash = blob.blob_hash
if blob.get_is_verified() or not blob.is_writeable():
return 0, self
try:
self._blob_bytes_received = 0
self.blob, self.writer = blob, blob.get_blob_writer(self.peer_address, self.peer_port)
self._response_fut = asyncio.Future(loop=self.loop)
return await self._download_blob()
except OSError:
# i'm not sure how to fix this race condition - jack
log.warning("race happened downloading %s from %s:%s", blob_hash, self.peer_address, self.peer_port)
# return self._blob_bytes_received, self.transport
raise
except asyncio.TimeoutError:
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
self.close()
return self._blob_bytes_received, None
except asyncio.CancelledError:
self.close()
raise
finally:
if self.writer and not self.writer.closed():
self.writer.close_handle()
self.writer = None
def connection_made(self, transport: asyncio.Transport):
addr = transport.get_extra_info('peername')
self.peer_address, self.peer_port = addr[0], addr[1]
self.transport = transport
if self.connection_manager:
self.connection_manager.connection_made(f"{self.peer_address}:{self.peer_port}")
log.debug("connection made to %s:%i", self.peer_address, self.peer_port)
def connection_lost(self, exc):
if self.connection_manager:
self.connection_manager.outgoing_connection_lost(f"{self.peer_address}:{self.peer_port}")
log.debug("connection lost to %s:%i (reason: %s, %s)", self.peer_address, self.peer_port, str(exc),
str(type(exc)))
self.close()
@cache_concurrent
async def request_blob(loop: asyncio.AbstractEventLoop, blob: Optional['AbstractBlob'], address: str,
tcp_port: int, peer_connect_timeout: float, blob_download_timeout: float,
connected_protocol: Optional['BlobExchangeClientProtocol'] = None,
connection_id: int = 0, connection_manager: Optional['ConnectionManager'] = None)\
-> typing.Tuple[int, Optional['BlobExchangeClientProtocol']]:
"""
Returns [<amount of bytes received>, <client protocol if connected>]
"""
protocol = connected_protocol
if not connected_protocol or not connected_protocol.transport or connected_protocol.transport.is_closing():
connected_protocol = None
protocol = BlobExchangeClientProtocol(
loop, blob_download_timeout, connection_manager
)
else:
log.debug("reusing connection for %s:%d", address, tcp_port)
try:
if not connected_protocol:
await asyncio.wait_for(loop.create_connection(lambda: protocol, address, tcp_port),
peer_connect_timeout, loop=loop)
connected_protocol = protocol
if blob is None or blob.get_is_verified() or not blob.is_writeable():
# blob is None happens when we are just opening a connection
# file exists but not verified means someone is writing right now, give it time, come back later
return 0, connected_protocol
return await connected_protocol.download_blob(blob)
except (asyncio.TimeoutError, ConnectionRefusedError, ConnectionAbortedError, OSError):
return 0, None
+147
View File
@@ -0,0 +1,147 @@
import asyncio
import typing
import logging
from lbry.utils import cache_concurrent
from lbry.blob_exchange.client import request_blob
if typing.TYPE_CHECKING:
from lbry.conf import Config
from lbry.dht.node import Node
from lbry.dht.peer import KademliaPeer
from lbry.blob.blob_manager import BlobManager
from lbry.blob.blob_file import AbstractBlob
from lbry.blob_exchange.client import BlobExchangeClientProtocol
log = logging.getLogger(__name__)
class BlobDownloader:
BAN_FACTOR = 2.0 # fixme: when connection manager gets implemented, move it out from here
def __init__(self, loop: asyncio.AbstractEventLoop, config: 'Config', blob_manager: 'BlobManager',
peer_queue: asyncio.Queue):
self.loop = loop
self.config = config
self.blob_manager = blob_manager
self.peer_queue = peer_queue
self.active_connections: typing.Dict['KademliaPeer', asyncio.Task] = {} # active request_blob calls
self.ignored: typing.Dict['KademliaPeer', int] = {}
self.scores: typing.Dict['KademliaPeer', int] = {}
self.failures: typing.Dict['KademliaPeer', int] = {}
self.connection_failures: typing.Set['KademliaPeer'] = set()
self.connections: typing.Dict['KademliaPeer', 'BlobExchangeClientProtocol'] = {}
self.is_running = asyncio.Event(loop=self.loop)
def should_race_continue(self, blob: 'AbstractBlob'):
max_probes = self.config.max_connections_per_download * (1 if self.connections else 10)
if len(self.active_connections) >= max_probes:
return False
return not (blob.get_is_verified() or not blob.is_writeable())
async def request_blob_from_peer(self, blob: 'AbstractBlob', peer: 'KademliaPeer', connection_id: int = 0,
just_probe: bool = False):
if blob.get_is_verified():
return
start = self.loop.time()
bytes_received, protocol = await request_blob(
self.loop, blob if not just_probe else None, peer.address, peer.tcp_port, self.config.peer_connect_timeout,
self.config.blob_download_timeout, connected_protocol=self.connections.get(peer),
connection_id=connection_id, connection_manager=self.blob_manager.connection_manager
)
if not bytes_received and not protocol and peer not in self.connection_failures:
self.connection_failures.add(peer)
if not protocol and peer not in self.ignored:
self.ignored[peer] = self.loop.time()
log.debug("drop peer %s:%i", peer.address, peer.tcp_port)
self.failures[peer] = self.failures.get(peer, 0) + 1
if peer in self.connections:
del self.connections[peer]
elif protocol:
log.debug("keep peer %s:%i", peer.address, peer.tcp_port)
self.failures[peer] = 0
self.connections[peer] = protocol
elapsed = self.loop.time() - start
self.scores[peer] = bytes_received / elapsed if bytes_received and elapsed else 1
async def new_peer_or_finished(self):
active_tasks = list(self.active_connections.values()) + [asyncio.sleep(1)]
await asyncio.wait(active_tasks, loop=self.loop, return_when='FIRST_COMPLETED')
def cleanup_active(self):
if not self.active_connections and not self.connections:
self.clearbanned()
to_remove = [peer for (peer, task) in self.active_connections.items() if task.done()]
for peer in to_remove:
del self.active_connections[peer]
def clearbanned(self):
now = self.loop.time()
self.ignored = {
peer: when for (peer, when) in self.ignored.items()
if (now - when) < min(30.0, (self.failures.get(peer, 0) ** self.BAN_FACTOR))
}
@cache_concurrent
async def download_blob(self, blob_hash: str, length: typing.Optional[int] = None,
connection_id: int = 0) -> 'AbstractBlob':
blob = self.blob_manager.get_blob(blob_hash, length)
if blob.get_is_verified():
return blob
self.is_running.set()
tried_for_this_blob: typing.Set['KademliaPeer'] = set()
try:
while not blob.get_is_verified() and self.is_running.is_set():
batch: typing.Set['KademliaPeer'] = set(self.connections.keys())
while not self.peer_queue.empty():
batch.update(self.peer_queue.get_nowait())
log.debug(
"%s running, %d peers, %d ignored, %d active, %s connections", blob_hash[:6],
len(batch), len(self.ignored), len(self.active_connections), len(self.connections)
)
re_add: typing.Set['KademliaPeer'] = set()
for peer in sorted(batch, key=lambda peer: self.scores.get(peer, 0), reverse=True):
if peer in self.ignored:
continue
if peer in tried_for_this_blob:
continue
if peer in self.active_connections:
if peer not in re_add:
re_add.add(peer)
continue
if not self.should_race_continue(blob):
break
log.debug("request %s from %s:%i", blob_hash[:8], peer.address, peer.tcp_port)
t = self.loop.create_task(self.request_blob_from_peer(blob, peer, connection_id))
self.active_connections[peer] = t
tried_for_this_blob.add(peer)
if not re_add:
self.peer_queue.put_nowait(list(batch))
await self.new_peer_or_finished()
self.cleanup_active()
log.debug("downloaded %s", blob_hash[:8])
return blob
finally:
blob.close()
if self.loop.is_running():
self.loop.call_soon(self.cleanup_active)
def close(self):
self.connection_failures.clear()
self.scores.clear()
self.ignored.clear()
self.is_running.clear()
for protocol in self.connections.values():
protocol.close()
async def download_blob(loop, config: 'Config', blob_manager: 'BlobManager', node: 'Node',
blob_hash: str) -> 'AbstractBlob':
search_queue = asyncio.Queue(loop=loop, maxsize=config.max_connections_per_download)
search_queue.put_nowait(blob_hash)
peer_queue, accumulate_task = node.accumulate_peers(search_queue)
downloader = BlobDownloader(loop, config, blob_manager, peer_queue)
try:
return await downloader.download_blob(blob_hash)
finally:
if accumulate_task and not accumulate_task.done():
accumulate_task.cancel()
downloader.close()
+282
View File
@@ -0,0 +1,282 @@
import typing
import json
import logging
log = logging.getLogger(__name__)
class BlobMessage:
key = ''
def to_dict(self) -> typing.Dict:
raise NotImplementedError()
class BlobPriceRequest(BlobMessage):
key = 'blob_data_payment_rate'
def __init__(self, blob_data_payment_rate: float, **kwargs) -> None:
self.blob_data_payment_rate = blob_data_payment_rate
def to_dict(self) -> typing.Dict:
return {
self.key: self.blob_data_payment_rate
}
class BlobPriceResponse(BlobMessage):
key = 'blob_data_payment_rate'
rate_accepted = 'RATE_ACCEPTED'
rate_too_low = 'RATE_TOO_LOW'
rate_unset = 'RATE_UNSET'
def __init__(self, blob_data_payment_rate: str, **kwargs) -> None:
if blob_data_payment_rate not in (self.rate_accepted, self.rate_too_low, self.rate_unset):
raise ValueError(blob_data_payment_rate)
self.blob_data_payment_rate = blob_data_payment_rate
def to_dict(self) -> typing.Dict:
return {
self.key: self.blob_data_payment_rate
}
class BlobAvailabilityRequest(BlobMessage):
key = 'requested_blobs'
def __init__(self, requested_blobs: typing.List[str], lbrycrd_address: typing.Optional[bool] = True,
**kwargs) -> None:
assert len(requested_blobs) > 0
self.requested_blobs = requested_blobs
self.lbrycrd_address = lbrycrd_address
def to_dict(self) -> typing.Dict:
return {
self.key: self.requested_blobs,
'lbrycrd_address': self.lbrycrd_address
}
class BlobAvailabilityResponse(BlobMessage):
key = 'available_blobs'
def __init__(self, available_blobs: typing.List[str], lbrycrd_address: typing.Optional[str] = True,
**kwargs) -> None:
self.available_blobs = available_blobs
self.lbrycrd_address = lbrycrd_address
def to_dict(self) -> typing.Dict:
d = {
self.key: self.available_blobs
}
if self.lbrycrd_address:
d['lbrycrd_address'] = self.lbrycrd_address
return d
class BlobDownloadRequest(BlobMessage):
key = 'requested_blob'
def __init__(self, requested_blob: str, **kwargs) -> None:
self.requested_blob = requested_blob
def to_dict(self) -> typing.Dict:
return {
self.key: self.requested_blob
}
class BlobDownloadResponse(BlobMessage):
key = 'incoming_blob'
def __init__(self, **response: typing.Dict) -> None:
incoming_blob = response[self.key]
self.error = None
self.incoming_blob = None
if 'error' in incoming_blob:
self.error = incoming_blob['error']
else:
self.incoming_blob = {'blob_hash': incoming_blob['blob_hash'], 'length': incoming_blob['length']}
self.length = None if not self.incoming_blob else self.incoming_blob['length']
self.blob_hash = None if not self.incoming_blob else self.incoming_blob['blob_hash']
def to_dict(self) -> typing.Dict:
return {
self.key if not self.error else 'error': self.incoming_blob or self.error,
}
class BlobPaymentAddressRequest(BlobMessage):
key = 'lbrycrd_address'
def __init__(self, lbrycrd_address: str, **kwargs) -> None:
self.lbrycrd_address = lbrycrd_address
def to_dict(self) -> typing.Dict:
return {
self.key: self.lbrycrd_address
}
class BlobPaymentAddressResponse(BlobPaymentAddressRequest):
pass
class BlobErrorResponse(BlobMessage):
key = 'error'
def __init__(self, error: str, **kwargs) -> None:
self.error = error
def to_dict(self) -> typing.Dict:
return {
self.key: self.error
}
blob_request_types = typing.Union[BlobPriceRequest, BlobAvailabilityRequest, BlobDownloadRequest, # pylint: disable=invalid-name
BlobPaymentAddressRequest]
blob_response_types = typing.Union[BlobPriceResponse, BlobAvailabilityResponse, BlobDownloadResponse, # pylint: disable=invalid-name
BlobErrorResponse, BlobPaymentAddressResponse]
def _parse_blob_response(response_msg: bytes) -> typing.Tuple[typing.Optional[typing.Dict], bytes]:
# scenarios:
# <json>
# <blob bytes>
# <json><blob bytes>
curr_pos = 0
while True:
next_close_paren = response_msg.find(b'}', curr_pos)
if next_close_paren == -1:
return None, response_msg
curr_pos = next_close_paren + 1
try:
response = json.loads(response_msg[:curr_pos])
except ValueError:
continue
possible_response_keys = {
BlobPaymentAddressResponse.key,
BlobAvailabilityResponse.key,
BlobPriceResponse.key,
BlobDownloadResponse.key
}
if isinstance(response, dict) and response.keys():
if set(response.keys()).issubset(possible_response_keys):
return response, response_msg[curr_pos:]
return None, response_msg
class BlobRequest:
def __init__(self, requests: typing.List[blob_request_types]) -> None:
self.requests = requests
def to_dict(self):
d = {}
for request in self.requests:
d.update(request.to_dict())
return d
def _get_request(self, request_type: blob_request_types):
request = tuple(filter(lambda r: type(r) == request_type, self.requests)) # pylint: disable=unidiomatic-typecheck
if request:
return request[0]
def get_availability_request(self) -> typing.Optional[BlobAvailabilityRequest]:
response = self._get_request(BlobAvailabilityRequest)
if response:
return response
def get_price_request(self) -> typing.Optional[BlobPriceRequest]:
response = self._get_request(BlobPriceRequest)
if response:
return response
def get_blob_request(self) -> typing.Optional[BlobDownloadRequest]:
response = self._get_request(BlobDownloadRequest)
if response:
return response
def get_address_request(self) -> typing.Optional[BlobPaymentAddressRequest]:
response = self._get_request(BlobPaymentAddressRequest)
if response:
return response
def serialize(self) -> bytes:
return json.dumps(self.to_dict()).encode()
@classmethod
def deserialize(cls, data: bytes) -> 'BlobRequest':
request = json.loads(data)
return cls([
request_type(**request)
for request_type in (BlobPriceRequest, BlobAvailabilityRequest, BlobDownloadRequest,
BlobPaymentAddressRequest)
if request_type.key in request
])
@classmethod
def make_request_for_blob_hash(cls, blob_hash: str) -> 'BlobRequest':
return cls(
[BlobAvailabilityRequest([blob_hash]), BlobPriceRequest(0.0), BlobDownloadRequest(blob_hash)]
)
class BlobResponse:
def __init__(self, responses: typing.List[blob_response_types], blob_data: typing.Optional[bytes] = None) -> None:
self.responses = responses
self.blob_data = blob_data
def to_dict(self):
d = {}
for response in self.responses:
d.update(response.to_dict())
return d
def _get_response(self, response_type: blob_response_types):
response = tuple(filter(lambda r: type(r) == response_type, self.responses)) # pylint: disable=unidiomatic-typecheck
if response:
return response[0]
def get_error_response(self) -> typing.Optional[BlobErrorResponse]:
error = self._get_response(BlobErrorResponse)
if error:
log.error(error)
return error
def get_availability_response(self) -> typing.Optional[BlobAvailabilityResponse]:
response = self._get_response(BlobAvailabilityResponse)
if response:
return response
def get_price_response(self) -> typing.Optional[BlobPriceResponse]:
response = self._get_response(BlobPriceResponse)
if response:
return response
def get_blob_response(self) -> typing.Optional[BlobDownloadResponse]:
response = self._get_response(BlobDownloadResponse)
if response:
return response
def get_address_response(self) -> typing.Optional[BlobPaymentAddressResponse]:
response = self._get_response(BlobPaymentAddressResponse)
if response:
return response
def serialize(self) -> bytes:
return json.dumps(self.to_dict()).encode()
@classmethod
def deserialize(cls, data: bytes) -> 'BlobResponse':
response, extra = _parse_blob_response(data)
requests = []
if response:
requests.extend([
response_type(**response)
for response_type in (BlobPriceResponse, BlobAvailabilityResponse, BlobDownloadResponse,
BlobErrorResponse, BlobPaymentAddressResponse)
if response_type.key in response
])
return cls(requests, extra)
+186
View File
@@ -0,0 +1,186 @@
import asyncio
import binascii
import logging
import typing
from json.decoder import JSONDecodeError
from lbry.blob_exchange.serialization import BlobResponse, BlobRequest, blob_response_types
from lbry.blob_exchange.serialization import BlobAvailabilityResponse, BlobPriceResponse, BlobDownloadResponse, \
BlobPaymentAddressResponse
if typing.TYPE_CHECKING:
from lbry.blob.blob_manager import BlobManager
log = logging.getLogger(__name__)
# a standard request will be 295 bytes
MAX_REQUEST_SIZE = 1200
class BlobServerProtocol(asyncio.Protocol):
def __init__(self, loop: asyncio.AbstractEventLoop, blob_manager: 'BlobManager', lbrycrd_address: str,
idle_timeout: float = 30.0, transfer_timeout: float = 60.0):
self.loop = loop
self.blob_manager = blob_manager
self.idle_timeout = idle_timeout
self.transfer_timeout = transfer_timeout
self.server_task: typing.Optional[asyncio.Task] = None
self.started_listening = asyncio.Event(loop=self.loop)
self.buf = b''
self.transport: typing.Optional[asyncio.Transport] = None
self.lbrycrd_address = lbrycrd_address
self.peer_address_and_port: typing.Optional[str] = None
self.started_transfer = asyncio.Event(loop=self.loop)
self.transfer_finished = asyncio.Event(loop=self.loop)
self.close_on_idle_task: typing.Optional[asyncio.Task] = None
async def close_on_idle(self):
while self.transport:
try:
await asyncio.wait_for(self.started_transfer.wait(), self.idle_timeout, loop=self.loop)
except asyncio.TimeoutError:
log.debug("closing idle connection from %s", self.peer_address_and_port)
return self.close()
self.started_transfer.clear()
await self.transfer_finished.wait()
self.transfer_finished.clear()
def close(self):
if self.transport:
self.transport.close()
def connection_made(self, transport):
self.transport = transport
self.close_on_idle_task = self.loop.create_task(self.close_on_idle())
self.peer_address_and_port = "%s:%i" % self.transport.get_extra_info('peername')
self.blob_manager.connection_manager.connection_received(self.peer_address_and_port)
log.debug("received connection from %s", self.peer_address_and_port)
def connection_lost(self, exc: typing.Optional[Exception]) -> None:
log.debug("lost connection from %s", self.peer_address_and_port)
self.blob_manager.connection_manager.incoming_connection_lost(self.peer_address_and_port)
self.transport = None
if self.close_on_idle_task and not self.close_on_idle_task.done():
self.close_on_idle_task.cancel()
self.close_on_idle_task = None
def send_response(self, responses: typing.List[blob_response_types]):
to_send = []
while responses:
to_send.append(responses.pop())
serialized = BlobResponse(to_send).serialize()
self.transport.write(serialized)
self.blob_manager.connection_manager.sent_data(self.peer_address_and_port, len(serialized))
async def handle_request(self, request: BlobRequest):
addr = self.transport.get_extra_info('peername')
peer_address, peer_port = addr
responses = []
address_request = request.get_address_request()
if address_request:
responses.append(BlobPaymentAddressResponse(lbrycrd_address=self.lbrycrd_address))
availability_request = request.get_availability_request()
if availability_request:
responses.append(BlobAvailabilityResponse(available_blobs=list(set(
filter(lambda blob_hash: blob_hash in self.blob_manager.completed_blob_hashes,
availability_request.requested_blobs)
))))
price_request = request.get_price_request()
if price_request:
responses.append(BlobPriceResponse(blob_data_payment_rate='RATE_ACCEPTED'))
download_request = request.get_blob_request()
if download_request:
blob = self.blob_manager.get_blob(download_request.requested_blob)
if blob.get_is_verified():
incoming_blob = {'blob_hash': blob.blob_hash, 'length': blob.length}
responses.append(BlobDownloadResponse(incoming_blob=incoming_blob))
self.send_response(responses)
blob_hash = blob.blob_hash[:8]
log.debug("send %s to %s:%i", blob_hash, peer_address, peer_port)
self.started_transfer.set()
try:
sent = await asyncio.wait_for(blob.sendfile(self), self.transfer_timeout, loop=self.loop)
if sent and sent > 0:
self.blob_manager.connection_manager.sent_data(self.peer_address_and_port, sent)
log.info("sent %s (%i bytes) to %s:%i", blob_hash, sent, peer_address, peer_port)
else:
self.close()
log.debug("stopped sending %s to %s:%i", blob_hash, peer_address, peer_port)
return
except (OSError, ValueError, asyncio.TimeoutError) as err:
if isinstance(err, asyncio.TimeoutError):
log.debug("timed out sending blob %s to %s", blob_hash, peer_address)
else:
log.warning("could not read blob %s to send %s:%i", blob_hash, peer_address, peer_port)
self.close()
return
finally:
self.transfer_finished.set()
else:
log.info("don't have %s to send %s:%i", blob.blob_hash[:8], peer_address, peer_port)
if responses and not self.transport.is_closing():
self.send_response(responses)
def data_received(self, data):
request = None
if len(self.buf) + len(data or b'') >= MAX_REQUEST_SIZE:
log.warning("request from %s is too large", self.peer_address_and_port)
self.close()
return
if data:
self.blob_manager.connection_manager.received_data(self.peer_address_and_port, len(data))
_, separator, remainder = data.rpartition(b'}')
if not separator:
self.buf += data
return
try:
request = BlobRequest.deserialize(self.buf + data)
self.buf = remainder
except JSONDecodeError:
log.error("request from %s is not valid json (%i bytes): %s", self.peer_address_and_port,
len(self.buf + data), '' if not data else binascii.hexlify(self.buf + data).decode())
self.close()
return
if not request.requests:
log.error("failed to decode request from %s (%i bytes): %s", self.peer_address_and_port,
len(self.buf + data), '' if not data else binascii.hexlify(self.buf + data).decode())
self.close()
return
self.loop.create_task(self.handle_request(request))
class BlobServer:
def __init__(self, loop: asyncio.AbstractEventLoop, blob_manager: 'BlobManager', lbrycrd_address: str,
idle_timeout: float = 30.0, transfer_timeout: float = 60.0):
self.loop = loop
self.blob_manager = blob_manager
self.server_task: typing.Optional[asyncio.Task] = None
self.started_listening = asyncio.Event(loop=self.loop)
self.lbrycrd_address = lbrycrd_address
self.idle_timeout = idle_timeout
self.transfer_timeout = transfer_timeout
self.server_protocol_class = BlobServerProtocol
def start_server(self, port: int, interface: typing.Optional[str] = '0.0.0.0'):
if self.server_task is not None:
raise Exception("already running")
async def _start_server():
server = await self.loop.create_server(
lambda: self.server_protocol_class(self.loop, self.blob_manager, self.lbrycrd_address,
self.idle_timeout, self.transfer_timeout),
interface, port
)
self.started_listening.set()
log.info("Blob server listening on TCP %s:%i", interface, port)
async with server:
await server.serve_forever()
self.server_task = self.loop.create_task(_start_server())
def stop_server(self):
if self.server_task:
self.server_task.cancel()
self.server_task = None
log.info("Stopped blob server")
+4
View File
@@ -0,0 +1,4 @@
from .ledger import Ledger, RegTestLedger, TestNetLedger, ledger_class_from_name
from .transaction import Transaction, Output, Input
from .bcd_data_stream import BCDataStream
from .dewies import dewies_to_lbc, lbc_to_dewies, dict_values_to_lbc
+125
View File
@@ -0,0 +1,125 @@
import struct
from io import BytesIO
class BCDataStream:
def __init__(self, data=None, fp=None):
self.data = fp or BytesIO(data)
def tell(self):
return self.data.tell()
def reset(self):
self.data.seek(0)
def get_bytes(self):
return self.data.getvalue()
def read(self, size):
return self.data.read(size)
def write(self, data):
self.data.write(data)
def write_many(self, many):
self.data.writelines(many)
def read_string(self):
return self.read(self.read_compact_size())
def write_string(self, s):
self.write_compact_size(len(s))
self.write(s)
def read_compact_size(self):
size = self.read_uint8()
if size < 253:
return size
if size == 253:
return self.read_uint16()
if size == 254:
return self.read_uint32()
if size == 255:
return self.read_uint64()
def write_compact_size(self, size):
if size < 253:
self.write_uint8(size)
elif size <= 0xFFFF:
self.write_uint8(253)
self.write_uint16(size)
elif size <= 0xFFFFFFFF:
self.write_uint8(254)
self.write_uint32(size)
else:
self.write_uint8(255)
self.write_uint64(size)
def read_boolean(self):
return self.read_uint8() != 0
def write_boolean(self, val):
return self.write_uint8(1 if val else 0)
int8 = struct.Struct('b')
uint8 = struct.Struct('B')
int16 = struct.Struct('<h')
uint16 = struct.Struct('<H')
int32 = struct.Struct('<i')
uint32 = struct.Struct('<I')
int64 = struct.Struct('<q')
uint64 = struct.Struct('<Q')
def _read_struct(self, fmt):
value = self.read(fmt.size)
if value:
return fmt.unpack(value)[0]
def read_int8(self):
return self._read_struct(self.int8)
def read_uint8(self):
return self._read_struct(self.uint8)
def read_int16(self):
return self._read_struct(self.int16)
def read_uint16(self):
return self._read_struct(self.uint16)
def read_int32(self):
return self._read_struct(self.int32)
def read_uint32(self):
return self._read_struct(self.uint32)
def read_int64(self):
return self._read_struct(self.int64)
def read_uint64(self):
return self._read_struct(self.uint64)
def write_int8(self, val):
self.write(self.int8.pack(val))
def write_uint8(self, val):
self.write(self.uint8.pack(val))
def write_int16(self, val):
self.write(self.int16.pack(val))
def write_uint16(self, val):
self.write(self.uint16.pack(val))
def write_int32(self, val):
self.write(self.int32.pack(val))
def write_uint32(self, val):
self.write(self.uint32.pack(val))
def write_int64(self, val):
self.write(self.int64.pack(val))
def write_uint64(self, val):
self.write(self.uint64.pack(val))
+60
View File
@@ -0,0 +1,60 @@
import struct
from typing import NamedTuple, List
from chiabip158 import PyBIP158 # pylint: disable=no-name-in-module
from lbry.crypto.hash import double_sha256
from lbry.blockchain.transaction import Transaction
from lbry.blockchain.bcd_data_stream import BCDataStream
ZERO_BLOCK = bytes((0,)*32)
def create_address_filter(address_hashes: List[bytes]) -> bytes:
return bytes(PyBIP158([bytearray(a) for a in address_hashes]).GetEncoded())
def get_address_filter(address_filter: bytes) -> PyBIP158:
return PyBIP158(bytearray(address_filter))
class Block(NamedTuple):
height: int
version: int
file_number: int
block_hash: bytes
prev_block_hash: bytes
merkle_root: bytes
claim_trie_root: bytes
timestamp: int
bits: int
nonce: int
txs: List[Transaction]
@staticmethod
def from_data_stream(stream: BCDataStream, height: int, file_number: int):
header = stream.data.read(112)
version, = struct.unpack('<I', header[:4])
timestamp, bits, nonce = struct.unpack('<III', header[100:112])
tx_count = stream.read_compact_size()
return Block(
height=height,
version=version,
file_number=file_number,
block_hash=double_sha256(header),
prev_block_hash=header[4:36],
merkle_root=header[36:68],
claim_trie_root=header[68:100][::-1],
timestamp=timestamp,
bits=bits,
nonce=nonce,
txs=[
Transaction(height=height, position=i, timestamp=timestamp).deserialize(stream)
for i in range(tx_count)
]
)
@property
def is_first_block(self):
return self.prev_block_hash == ZERO_BLOCK
+743
View File
@@ -0,0 +1,743 @@
HASHES = {
0: 'bf3ff54138625c56737509f080e7e7f3c55972f0f80e684f8e25d2ad83bedbe2',
1000: '4ec1f9aebc8f7f75d5d05430d1512e38598188d56a8b51510c9e47656c4ffad9',
2000: '5d5965e43d187b6f8b35b4be51099d9ec6f7b0446dac778f30040b8371b621a2',
3000: 'd429f69a9dd890f7d9827232a348fe5120371ed402baf53c383e860081f6706e',
4000: 'd18fef650f23032e4e21299b8b71127f53ff6d2114c3eac4592a71f550dc5071',
5000: '12c1dd1b9cda29eb4448e619a4099c7bb6612159e1cd9765fbbabeee50f1daff',
6000: 'ebda54cab7979bc60f46cab8d36b6fcea31a44c38d6e59bdcb582054b6885cf8',
7000: 'd2be08791053336ae9bd322ee5b97e9920764858f173a9e0e8fb54ee5d8bad0e',
8000: '94583639d9e2ce5eac78c08ef662190830263ea13468453b0a8492c3aca562fe',
9000: 'a6ebb24a3ec9bb3fbdb9d9525729513dd705fe85093c33a07bbfe34ba1f1eae0',
10000: 'a5d32c6cd43097f3725e36dd55d08ec04cbe62e40d1fb3b798167cc9c1fa1ce4',
11000: 'b5cf8df354bc26e8b0315e8038564a9a7fc1faa6ad75e67819b7eafbcfdcb145',
12000: 'ae7b0e5106299e43b78f337fe15e85c27f88d5bcf618a21c03ce43c094dde909',
13000: 'b57dfe631571b654e11fb4295aecfc11f612664d22d64a9690f64c0b3ec128bf',
14000: '76e55cc1c38d9a6a349c9a42ae0b748d0d8b27ebb30539be8bb92e71e7abca17',
15000: '5d33cbc872acb824a89ef601aa3c918e7f4dbb8d5be509a400fa41b03f7fe103',
16000: '25760f6a6bfb58104d02e60efa62ba5ae141bbac1f65d944a667ad66fbef15a9',
17000: 'ad37d98bb272aee3a950679cdd4cc203b1c129bb3a03fe74223ff858badfcc50',
18000: '9eac34fc453b97b1c62228740efcae4ef910b0bea26c322e17e3474b81713b0d',
19000: '06783429c8f50cc6495da29d38c86c3873800cf1eceffedfffe2a73f116a39d3',
20000: '0237c620f7f68d9ae9852d00551c98a938352d2e3d9336c536f1cbf5246fade6',
21000: 'c4dbb4eea7195a2cf73de2d49b32447cec9c063051acbd300b26399c9242b0ed',
22000: 'd18fea80b659d9690830c2f37c7c9d53f79fb919debb27cc2131044df26d7ff5',
23000: '0371254f4698d3c46596c1fd861e2e1e75053beb69c3c4d8ee6702f0992f0026',
24000: 'dbc97c3986bb9bb6230321d1fc8b7ca13369033f57b08ac01bb775f7241861fe',
25000: 'cfb9baa125e91d02c060940b5b51cce52b495a0b0bd081cd869a7ca45a0e0585',
26000: '6e07fecd3d0f72b8cd50a3cdc1e9f624beb3dac9f6356308a1ec295f47d68f3b',
27000: '8ad248da3aac7c1db958d0ce8a2d50fde461bbb96a628efa215116d3b74b74e5',
28000: '4e8231ede23b397c49f1a97020ab5fe45465936cb6c8cf1c39ade1b38a80547f',
29000: '1453b7a4be4076e2c77b94bc35f639c571dc05b225a5f8018a9a60337fb3ef66',
30000: '43d779c83032d75e8df09aeb6ab103ad41618777565fabc2e56bf31c3a20285a',
31000: 'ea1426f57f75ee781a6e569772944119f5e65c8b110b31a32b61d951c3a1900f',
32000: '43d1a3941b2c3033f78040830d39e657f74958d71b3cd0bc728550469cdb5fde',
33000: '3b0c0eac231476efed2b9bb9dff206a6438a14946c3b4238bf0af6d03f9a350f',
34000: 'a8136e9d6f3f5048ef8f9f56628009f5a87bb28664ba8028ecae0797e5328309',
35000: 'd8136e638995aaea03c6b50f1cc0f1891bd107290f12a6ee57e3cae36b7e3374',
36000: '06a7692cf5ce9bb71461e5427e6f55bce2904f9d3af4c28e7343fecbf0e336da',
37000: '12a1a49c237497295150f99c76d79d8affe9fd720e0cbffa62be80e2bda7f832',
38000: 'fcf3e526931b7f7cead47652f12cdf76641032bc545e22aa0dc83bef46a085fa',
39000: 'ffc34a198ab1cc13819c0c90b26a8455627ff12868b49ac1e6b5e1a086ed011f',
40000: 'ac71a9895999531350cf77a701035b0b59720a39994c990354b9cb1f6ea8eb49',
41000: '4fa753f5f2de41ad69ab32c6802eeb352f070d8684c3bd154eeb8a3c17aa6363',
42000: '6f0febbddf9248e3df2254a7196ec13f353c7a8049e098f2fcdf920580abffa4',
43000: 'e3e98488cb203be9b531e3b5ce2200f4e5655297a4f4534bcf486ad5e26687bf',
44000: 'c7c6078e30163204e49d493d506f72f71d68d2a591874328342ca3cca81c87bd',
45000: '2a65629529f69c15a6e404ba54b83e024acba9d6c786be554fb14d28951ddf9b',
46000: 'cbc148ad661e6fc390746e94cf21df71edcb222b4e8d8f0771237a8e03412f9c',
47000: 'd6977f770749b7ce1a624a61f7b2d26d817a27c50a895466ee10f8bd758c029a',
48000: 'df05fddcd5ccfab7811e5d7b6cf7b49df71c7e0f77a57ce384936948114bb93c',
49000: '9d26a67c06f066229bcdafd55cbe71e2b2518465d6deb618c418b2f229ef513f',
50000: '4ceb896d901315e0f3e10f3b1ed8472f5bb5230d4ed6377214b07e425e515f6a',
51000: '218019bd8440a8c242e84940119d4eccccd26134a208cbb92cdea3480069f482',
52000: 'c15f1538280e71abc1cf3e117c79b0b671a9f6fb46102fea7b9a1fbcad418011',
53000: '0ad4d3812a42e23fc0defe7c62dad451266ee21699ade625a5f00b91993850d3',
54000: 'f994cff7b8166b0c39a310bb686f043fe9355678ae406a7fa1a74d4bf6f9b849',
55000: '6723d30c99fb45513a42a3ffaa50d1b8350e3c8b52c6dc6894303147eba95d55',
56000: '947cbea79aabe2051522d9e957d92b86dee2c420c32a83cf6f6d655136731926',
57000: '8a153fa31a971e7e304650257bfdbaac230b36a9beebc01a85a7aab9a357a5f2',
58000: 'd19a5234ab995cb851d691b2c2c84ae665890621c52794e54c16d6a87f516c7b',
59000: 'f9c7a353e62ce57a5e3f983362b08fdbff5e2aaadce325bba7680104a6f51cb7',
60000: '9e1d535bd31525803e93e4b4c2346fcdfee7c5cdc4b9fe3032df9295aa85c754',
61000: '640387271e7eb0816b9e290dd157f58d1eb54cfc39bc7ecc4486099af2a22b76',
62000: '3d36acc4169a238657729df674277c6cf236e21c2bed4e99b8f1a6e82d00bbd6',
63000: '6d4201e45595ea160773d63bb1481f78664156de1f4564d9a8c3f61b3cc1a6c9',
64000: 'b2cadf2ba39b2d5e688029cbbc68cc6d71337b29861dee24fe941b31be1198a3',
65000: 'd8a364d385ef42e53ce88c82355235ee7a27dd3e642849dc22d08a4041695dfb',
66000: '7025d4b2d9537002ce2b9ec42c596eb864e30f41c06f02413997b5629d6dce3b',
67000: '852a3fecfb3a77bcd6d2604170cbe9646d9b889c90d234eddc8f9d112777fe41',
68000: '38956213012f66a89377f4914db838faa6738732f7a7a865ebc5777f2f126b73',
69000: '4c89b07d23727d4bd1a080bcb9a22dcdb45c904f4c9866153a000056d7ec1546',
70000: 'e4e7fbf0d17b5f933deba12e045ea23eabfa6f7dbdc633e5c3d3ef8ddd066b47',
71000: '1f07c36e82f8e7582ec9df23ddc90715df45978b6aa0a2f17c08ee14cbb23000',
72000: 'c59c660b5aca2fa3364524ef43a4fa93fc3f16df777427cf1d810f65ce42e4d2',
73000: '83a05ac2fcae8c5d50a90df308f641616fa9b9fb553763c874b764a73def2f95',
74000: '448d209c032a2c6f5330481bf34fcf0bf1ea2c47f43fa5f82cf42e0d8f4d2b20',
75000: '5627eb3491e1154ad2c90094da85dc7dbfaf89d3765848db1ed915b7e26dde58',
76000: 'b1417b1b2360e24fe42345508ba0a333f4bd76f48c959b8bb40a99bf34ec5ba0',
77000: '250aca2bc4e6da7a8e1cb2e4deb0396aee7c317177b099d7dab9731b4c0f7573',
78000: 'dbfb1dbe80a5db06c9dcc645c5279eaf40559f5e04df0e64b696e80aba5659e0',
79000: 'a7183a545925164b5a9882810ae117d0f3df9b3ed55885fe74ebea284bec484f',
80000: '5b4a7bfd9b5394e2759daecfe472ea6e92b281a7821b106314b2b2e0facbaab8',
81000: 'b7075fe530b44e4f5895521b3cf99a79fe23d8bedde0027d6e7924ca93793eab',
82000: '6a1fa75b896d799ae84046cba7cd942dad47ec04bf50a85cd1b2bb15861995e1',
83000: 'aa5f4caf970433e90e2e45f9524ab9a6e22281505ada4c206ecf4672486240e3',
84000: '103732d12ef792a1adbe8f295ca6abd008867323ee996698b2650bdc1bb8d06b',
85000: 'a214c16ffdc505d6caba7bd0c2bae766bb19f7eb4d436cfc037e93783afae7e8',
86000: 'f54226ae9a6814a345968b5c2982adf638b995ff50e27de1890b3760ead12158',
87000: '26546212a0720cab988194432f4fa7c3e47caf0fb8e31efdd1ee93c3ad056868',
88000: '48f7d47bc443bdbc6a37ffe8f1d0d91de16d4a470d03655692de8a04f84c2561',
89000: '005bac455ff093a39b907af0441897e7a9ceb19eaf51d9dd9ec2b5ea43ff6d57',
90000: 'b482f10315170e6bd280325e180ce37b0423b685baa06d33586ee659649b71a7',
91000: '56db4d5d6c460fa76d492d9900c02e8fa99bfe97c7d9fa2a75bf115128ea0da7',
92000: '72ac917c4f531f91f65fbe33e78b4d4e2bef18aab3d2ff584d59fed1ba2cf398',
93000: '1100df2fdb03d44530723d131dd7d687053db6a317ac9181cfdd51925152df02',
94000: '6d50f66efb571136501340fc00b52105e0011f0f11fd68c9b185717bd42f9307',
95000: 'bd0c996d9ec40d5e91c2b5f278a5c9c9ea0b839012a7d135e1afc73a725bd8d6',
96000: '4ebfe8a3e1a1625632896d5d2970d8b080f08c5948f0ac38e81f3fc85db5af5e',
97000: 'cbc03464aa3513aa33b540d0f026b93db7607c4bc245af8fd88c3b476ea5e394',
98000: 'c9a185ffa55d01fc73c2e60004b0c8dee60ffbac7784cd16046effd7a0a51a86',
99000: '1eba97d5ee69229f4cd697c934d60cf3262ebcdf4c79526cda71e07b52fa22f7',
100000: '0b1650207a55c64e7a6cec62ddc3cc190ed3796507bbd48358a0b7d6186bc3ce',
101000: '1f47841b0034d3cf1046660c2246902f679445b5ef621df2762bbea33b7d685a',
102000: '42f63b1b622d08e120b5405ba65d25e1a9777312cca77248b7827d7084e6d482',
103000: 'ef4861fbd4e578ff80e0d5e2afc62fb113aa203d3933f74efd3e2400d73de922',
104000: 'e97a3cb78c09eef732757e81c6a7135beb33d95398054b45df738853acedaaad',
105000: 'd12e8feec15893ccab662a1ad0754b2ccbc18b078ff3894eb2214ee4ac2f7af9',
106000: '83ae84103b37c93b2d1535fc47b053194270b3c186b1b25d69cd8a1540caaa1a',
107000: 'a102c6dce88bc4695250c24a0ebd2913b2b85c211b7d5dbbd18ecd95bea63144',
108000: 'a0c20b36140d55288dc4dfeb6c16a6ece3d43efb57b1728bc872c35d6660c704',
109000: '412fe428b2929c3cf5c6991a2657e5414a569b30e77a3f4fecef14239d89caca',
110000: '8518706e957f3ef892d1d2d7f65c5f4588620be994cda9cb7d81fb600554a456',
111000: '66f4e25f40d24360299e5358f954d8a79b245399c46dbc4e8a3db0c10fa14e18',
112000: '81c0ded5c5e30f92eec84ca293abb0165068d917bdcf436754a0e9b0ef1e325b',
113000: 'ebd5e034a45517e59e96e986b55e38051a5d2a3120a7ed92d4f3e01e18e73972',
114000: '681b85720d71ab660c7443392883ed4bdcf9aceffe202cc5ead94664e315a744',
115000: '3fbda839115bc6e6d451a05f78b27b35b137c94374fdaa42235f3035980e85e5',
116000: '7bfb88d39fe0ee7ce046648676b226997a812af1f1ed79040750045536c76067',
117000: '5853c6d48fbf99b06d3e1f474e1b667ca598712f9feaff8cda48a95935cfc498',
118000: '6a23987731379799289d2a527dd40fde8aaca8398625d2a2e3e646372ffc99e1',
119000: 'af73cbe957865b60c3c3139dd1e2bf8bf1cc504dd167545463105996609750c4',
120000: 'b309a20170421fc4ddfedfd7611716e1cf5a802a1db09b66b9ae448fdf958792',
121000: '624cc893801e4da093ca973e2135352bb428dc278e47f5308412d05acbbdeacc',
122000: '2f20c67ac3485b6e5407762a23e8a7df1bb380f1607f538cb5e2e1ffaad578ef',
123000: '89c73ed65ab8f729cdcd678dc52e876969907f1560898220702a696893744e00',
124000: '17d8155d97fa4e8705aa55969709f9cbc97f87815d40f5a2b5cf4d7b85dfe4b3',
125000: 'd3cbbdb7f51c8a1f5794bb3d1f40c413b313f91bd3f3dec0d8ebb6a02107e52a',
126000: 'bfd02d760eb5de2e792ea74215ae37d13174c3fc0aff08587281f0f39e427caf',
127000: '937dcc09a14d1de79f5eb679996225995cdce69497cff5f12c39c098c42656a0',
128000: '81d372b07e2467bfa62bb4b974d0eb75ea313e547b9d4e62f3c617f939bcf67d',
129000: '9b4b315738ab8b5266b5accada140dcf487b2f509f1d758b9b972b93ecd03c9e',
130000: '8ab3b1862cacf7f1c4f6617f7280ba31547dadd62bec9c37ecc8d074c90be2b0',
131000: '868d6520e526b8594c44a05eba67ec1de8e01b547b34b62ebc5dd09d656e9072',
132000: 'ce09471822965da31db3188cd156538c44337da2952486a34dd2ab372ea490a8',
133000: 'b0c34cd700630c9281c8657dd24baf609f72f897442538d77f4ac355199731c0',
134000: 'a76e5dc895406a3951d55b557272915810420bdc3ac076863c7efdef3b115890',
135000: '2b70fd7021d40f6df15f3f72bc8e034a8a0bcd8542847b5b7a2088253351b3a4',
136000: '6161ee84f6fe100a02de0675bc9500dc18ccaec74e7da4b92135f0ccee6fb663',
137000: 'ae5770a7158089ca9adab8bdd07f1259e6f204ed377af27fb6a772fe1c031864',
138000: 'f3b8de21370968a90cedfcb1cb6803ec9a6b7740a94d8e4c60a80407fd55a12e',
139000: 'f08f6431d7367b71b25b321809ebf48f797bf7fcc943ba366420fd3f3dc00e5b',
140000: 'e76345dbb8c4c5ed1ea37249f892f64098557700259fb360401a559c45909041',
141000: '776b333f5b221f6b443d6011bc8d0753eed5ce2446a6d3ee99a5800c7159fa92',
142000: '9453693cb846f27ba2ac39194e5b70cedfaf435b73b5f7f288dd81343ff605ea',
143000: '08aea64cc4eb0170d2704cbd3a4ee871d4f3e53a8c1395288c029eaddd1aa097',
144000: '801d60c225301481c761114d5152780d5002b77aed18845c32c9e3804a5db254',
145000: '78cc80d54933b84248aee32a18144d03237f948de7889c97e2112d17ac998009',
146000: '1e954e8f2fe59da3feef0514f6b271f52961467782b2530501c294a83181f176',
147000: 'c0ac42fe9c0e1d2b0b7e6e796c553eafbd5d7bbce7df84903dabe16dc56ba863',
148000: '75b5441fc1785dcba69b9409f24224ba3207d624bb0e7d54af24118d08c788d7',
149000: '20e53048515cc34507cf2d1dd160eabafaa97793d81626e9fce38d540142dbc4',
150000: '5ff7a08461fc3fbacf8c6261dbd34a790e271bbc39908c23d5417f0f7a71b71c',
151000: '5c14cab7670821966d5aa61f09b32d93c865764b6e258764a3c33a82a3133fe7',
152000: 'e595968e040b54ed6572862b4089e005238399014d99bf4c361cc5922dbfa6c0',
153000: '1292bac3fab11de09f46a8327511b7c37366d0f2a96a027f4bf2f16147a39af3',
154000: '3db31267722b500dd9b26f5cfc4f06f26c5c343d181464499d3bb4aaf13c3c20',
155000: 'b92962af73b42335125f28d7d3fb97e3aa6ccf47c3f4229eb7fa04b67fdb933e',
156000: 'fdbcf12415284d8cc0adbf16996b9d37a943fd1a924c5dee5f47dc8add23f278',
157000: '199781ca8e5929708f834d6f38ce3e2fe196ab519f9915f14401462273431e19',
158000: '969017fd15cf6a73683de287c5e19fbd69ebb5aaf7c13339297f7177a716de3d',
159000: '77c7995cf97218655c195fdac9010c599859a46f760d84750189114d2d2d1d9d',
160000: '00ff84f0d845b4f1099d970cc0c3e2dd1c2584c611449f2318a3f27327246d51',
161000: '08481dc5f61e776ae8be12236d595549abfa0be28b187d80dad573594d94c11f',
162000: '71497ab26b05453f3c7057f8bf57fcc8ba30920a6032ab0ae3abdd29e0677582',
163000: '08713e0b750233a3843241d24573c4800f32c894000b7815860048ef3e7a06db',
164000: '01fd806f1879285ad5234f38789074b0ea3a0b2707bc6b49aa9bd51ecd26ddcb',
165000: 'a990225a2f02a77c1c61d518d46dde1e1cff3f4df0ed24eb463b97f84c7a2616',
166000: '7772a601a6e765ee340eb666cd645503c9988430ba4f558961246e9e69349b25',
167000: '389c7a8979078b57d54936497680c7908b9f989e46631358f1c31a3094621557',
168000: '2e1df97fe3cba7add5f05734a119464ff7c22bf92701ed85642da856ec653aff',
169000: 'fe0adfe455f65cf90e1632ef39bf9ca5856020d995cb71ce8a4998a40eed5998',
170000: '87487f255873e9f6411565553fa9ccb9518e7baf71ade67492536278f1ba0feb',
171000: '37cf239bd630b7c891019adcefead4baf19a86b40e21f3dfe4a76a78f2985103',
172000: 'e2f4087b868558af51dc8f40d77c300d76eacab17e5fb8bbf3b1f99c02d995a7',
173000: '273f32717f740438e93063d259dd4f0f160077c34c937a7d6a9b1f9fbb34d87d',
174000: '807a861813ca690a6fc0715f354f36bf491c34588b5ab778c0f4e692ec3fc152',
175000: 'dd74cf3d686c59820885bab134c81b7079f2edca86a25944fe0aa36a96941550',
176000: 'aba8163b91a905aab5ad6e7599b919fba650c446938dc7b2352759203bea8957',
177000: '260bd0d053038a54565f9b25c3894cfa875c9426b9d6e9915ec1df03bcf90ac5',
178000: '7a527f71743c4dff25f13b918ad4a6d91fb0bf9c873a7282e51cb4edf545edbb',
179000: '945397818b80995ac4e23c785b6d5ed2eb01338a2d4ca6f0cf40986e87659d86',
180000: 'ba1b5263f5418844796a19db06a53c095ec0428696d8ee953790d6c86de0795a',
181000: '2b673859af43554accd8599bca53c2ca94500ecd65df7672a1e586595bdec7ff',
182000: '08bae9f096e348c8c6b42ac2762298ec3faeba896524968478fff1d40017b5b0',
183000: '211577ae9a4f93fa5dc3e33764c31048ee306b5e5ffa5081fabdf3f30e49a977',
184000: 'a4301eead637043a8751a30c13935af971a731e03e90317c5a16c1677e50d837',
185000: '1f4aa362281293e8328b6c5b32a9948ef03dcea81c6ba13f94531f0c5e42627a',
186000: '0c6b15f2164a1b55bdf0eee2b0f5ab7cb9d2dfea6fcc82686483aa7a3659f6a4',
187000: '5bfa261cad0e4d271814de1f4752352f35da10f24b8016ac0065172e1296052a',
188000: '94c2490bdafe70d65951e4034e5b9122fa6b376a5ad8c9e6276289c4a5e059e0',
189000: 'a0119c5d57523b7ccaf1e5c6671eae9f8562bbf2477f7dd8c6847c6da41707f6',
190000: '4d82802c2464053c391374eb6124f8839faa9b343bae3f34e00f87fad9f45f9b',
191000: 'd341f5b4d20894be520eeb533d56c5737ad02a7ab0c24b2463516ff04e3b8b0d',
192000: 'ddf1a72ba512786d39f7c313f710bbd452040f1eb8a0c0ee7a4b9192199fcc3c',
193000: '514e1b40e5e15dc72e75cabb8d490c5e76b968e6dcdb3acf518b47e3a2020407',
194000: 'd0bac7ca0636b662d6be169bb5af807c891523a44fceda68f08c8934d6e9e254',
195000: '00c37e81f0dfe63a8fdefd0ba91442019899d50f72b43c4de8017e767777a5af',
196000: '581a9d093458c0157464c42fa2e2d9a1f61f7edf1ccc5dd77a4fdfdd9a4b37b8',
197000: 'd7555fbf7671d2475e38f62418f6130ddfe85b8fd07fe8a9815849316165d401',
198000: 'ea388884ad5abb06ccb1d0d2202dfef5c4ce4de0f3040df089017d01f14b9530',
199000: '6a80cbb22620b7a5f99f4ea36380b7bcd22e87feaf4186bfba0b9aabcbcaab70',
200000: '7a7231293d242887c6dfa517a78b60d40b41456b6f2fe833c0217074664a61d7',
201000: '0d6bde8cdd0370964b83ee766891b3cda7b9eec098dfaa5ef05842b29bbd043c',
202000: '223fc091bddfbaedec92099f873fc1a6bc50556f3421f472a63977b3351b95c1',
203000: 'f87fb207fd397f8fb5c5c21f84f272cf33d840e20ccaf9de810af784d8459142',
204000: '6273b130d819d84492bc50fa443ca401df748e58fa84a3708a1797f116f60f1f',
205000: '498edfaaa9d5f0501aea5482ffdd9208b86af46fcea5a1cbc82d51d45eb6a256',
206000: 'c069138610493a70742918b5c624cbf9275bf55ee24649e32244238433883ea0',
207000: '5110744882ba11b98df95f11e100bc398a5aab4c8258133b518b5abb58c7fe64',
208000: 'e5690d4e0561ef015d6e2997df3b8b95163cd884d999d8c9ac212bd7e1f06ab0',
209000: '17a0a3bbf2fecbfa5c7667524cb2a452f49b8dd089af132f9d2c530ca0b677b1',
210000: '3a9489f3aca89576152949a67df6462d25bf3a78976451b5cf8bd8385a10e6ee',
211000: '45196e756c7c282c4a9e7133adb07f103d0f74ee90919cc8ad89ca3e5d38c6db',
212000: '590f21c83694c35028b9c9d632a082307b8bb27ec87e2c0c4e2881a7be9c6637',
213000: '46e56c45305c51e4d22927e449a1855064504cfb58cfceb0f293d8a4f1dca7cb',
214000: '50b3f1bb82c1b213ed2cfd2c1144b8e67e5fa731dafabfefe063f8658c5536d7',
215000: 'f53709db4ca6c12b80c64501eeb8f176211116b0fbc3a650082cf830792b206a',
216000: '6818db3f6582e71021a84fffe751fcb47db2f734d59547be47a373575b5c7d8e',
217000: '01f6f8a0699ac533ee7921f63334af07b04369d5e580c1c609c6fee64c8c6b8d',
218000: '81e9ff93cdf5686fc42663a056c74fb0f560c29a5d23f21ef92364735a865392',
219000: '80b32bb6bc3c54a19d3b604f56084b3bda13afd4af4130f791505e44fdad6c98',
220000: 'c427f24ab14449b1430c5302385f3a77f90f19715d940e8dfb7c054828270163',
221000: '4303a0e1c6f8489589290370a6843d97eb02a01be8c227d905cd0e30034feae6',
222000: 'ccfdc1768afc18cccdf2b5d90d40158d86b110d878955b6570ad9e6e40063008',
223000: '6baefb7061239c02472c4783de6d32936f7fa1da9fa6d2448b4d92f40b89e79d',
224000: 'bdc334f5d1a3bd1d19cf589e3b1ab0e8a324d44ea1c70297ff2de6a66c99399f',
225000: '82d29612f96c55dbe83655db68afe39146aff1418af929f05033fcbe612ebcb0',
226000: '179b4144f4d644f546f001273c1e5382a8de50f84ec06a2c83b02808a9167d28',
227000: 'a350e96895efe979223475151305eb8d1739121b566964033b9967031ee5581c',
228000: 'd208c66a9ccb1afde8d1540d944bfb43522015f9d2df50851e66ce1b84b06c0d',
229000: '95cfabe5d18dd8d8e4a57b7bf7052ab09b7138ba0b3c9d2efdb32352bfcca149',
230000: '3e399d08c33b8fddeb687deb276afd3ce101d19f5a71b232532951f05d46d8f6',
231000: '91c0c17d5df1c553bd7dd6db8feb58a4436d598ef4acb1ed9a06da65f3cca83f',
232000: 'eb3928a522dddde176cd4c68fa95f1e953d74f3759af4b030f7e2be4c466c45f',
233000: '3ac946df5a319d31716f50c752a5769471c0f3ba4b1002613996981c8f32fa3b',
234000: '4c9cdbce1f2a4324368130c154e2d24e885877c13a126a847059ccc289fc922c',
235000: '8701d98b254fd0b588f022ea140e936d496ca8ccc0890ce133523432f4c09e6a',
236000: '0d2d45adcdf950541c25ff5b2b87dff5faad479094b1f68804a62c3151e6d598',
237000: 'b8924ead720d0e7a304c45c209ef3fdb90cde437d138bfed2f1e8357fb3d951d',
238000: '4c9c62a3b3f6bbfb08edc23d63f9b35ed94bb38393fd1761f9b9b810bb72f68d',
239000: 'd01494b4999c2c4def2564eeb1207fd1f3d35a62fba2e61b386bfab498cb9b6f',
240000: '3021d8f0edbc0cd2ab87bad5feffac54924b51eac9f6fae8ebe289f474f11a8c',
241000: 'd2ecf0170093d3a2dfc876f673e565e3e4aa76d23896af51c96c004cb14ce67a',
242000: '2b5ef67326178702a1a35fe4a848d22b791114b59121e41a25ddc1bfac82c6bd',
243000: 'b2d546f127252c3fa5f7ca45c61268dbaef66027484a1bcdadd281fabf97d34f',
244000: '58f9b398b871dd62952899d31661c118ca4f0d1c71dd165523b889d62b154393',
245000: '504760f0e46e2de0a2c8b4dd3e93c19d0f3e0e1b36916f02cbff166655816206',
246000: '15ffb23b844e6ac648a7fd099be8e631cb51b3a2496c67ac2dbedc4f6818d7a7',
247000: 'e3e56e8575552202cf35fc883bfcfd92cacf8013a6536729ec19e53bdb0db128',
248000: '22f75d137c7fe42a80ee04a386fdea8cad56e7ce8601234b6ec676498edf9e34',
249000: 'f84709269c393cae8106ddb1fa3a48cf3226e97bcc108717e483493159921aa1',
250000: 'f3aee545330f18ec49f6f880d7f9fd31ed6633279881dd13177b0a687c51b000',
251000: 'c2e6a4cb3c6e88f523db2ac24fc7fff3b8a27d13ea7b5d614806ed96247025e4',
252000: 'efba2c1f91a13ee8d7a8658e70893ccd3014a7019c225e356c3ee67955b58756',
253000: '9097ecbbc1279bfa2c92822a6485714ca7ba35cd3dad9ccaff5a8d24645eae92',
254000: '00a41b720ec0d7a9fea3229b16ad8d065cd6bb1d14587eb3404de33e45db9454',
255000: '7523636406176d391928c638e15e959a9d0255aa71dc9f3bea8995cae02d1d51',
256000: '8d3f7c5ded24e2488e25d6030d78d351354734b8b058caf6143430e96ebd5d0c',
257000: '48fbbc3737a718163a501e52964eb58aaf4163aab7ceaa4f75f928b25c376ec2',
258000: 'e1052e7ebf83246770a740c0c6ce8f011fc7007448777c533d67fec810fe5e64',
259000: '043f7d98b500fafc248549c1f5a9727fb0cec4e3e5f13071eeba459fb9179d13',
260000: 'ed2f3960b077651ac554b277dd271e12e6cd05942fb54b9464f18432adbe5ca2',
261000: '234f17bf2054ad4d3efeae165c3359a14e08d2e53506c839cbed6ba3d2a48ab5',
262000: '4342d662c4dab581bf1c123e057e4413bbf74b7a2a331e80f7099201539be54c',
263000: 'a7b1380371fbfe64bfb94bed5ae256f7db59a4661e461221cae99dcd54c72a8a',
264000: '4f2a606cdc5f346e0855d962cbef7c2e4993b2b07b86ce658a796b1baf6179d8',
265000: 'cc09cde8c101eb17b5457bf976525940904cb839e70ef3335780eb0261976826',
266000: '3c5f24533723afb124b2bc8c2f8d373eb5640dc4ea4dc76b841e81706dd6b29d',
267000: '467c5ca63621be011a07129d0c3ea785e2a0853cc3c0d1e9cace122831d42507',
268000: '07df13a794f6c0fc77142fbd0afa23121ee2a2a23bc9abca35c164edb5f6ad1c',
269000: 'e2be277e83a997a111fb45948e9a7afdd7925feb1708a84c62968cf6a42da1c1',
270000: '463425dddeeac2f27ee717364073229c333a7ba357888623c3b15a1db52542f4',
271000: 'd62e0bd8584fae7e4893c18252ae8d1c245e2e4c8e2f2d73703c681c6259f19f',
272000: '85cabf5254b6b6292c105e68d44f78d6effeec582467caf9d30a5deeb43b6a6a',
273000: 'f5509cee847a757c84adea8e40f3c7c0cc6a4da33a8cd3019cf9ca850d0bc93b',
274000: '2cfdb3477de2fe1c81c7133094094c79e933cfe6034c158de07ffd0c4c08b564',
275000: '1c1b8e447e95ec3c306fc68528631682bd85e1c876d86223fea28b7ff383c16a',
276000: 'b37e5e4f6d29590c9a58773a4d4384cf7c51d04fede4a602ba8095e14bf65993',
277000: 'b92d6ffdecdca014c946c2fe8c6c404ccf60d8401e7631918da8885bd360daca',
278000: 'c1f82e49c8f66aa580a216c8e08262080ff9fc4f4df1c482ce8b2695b90b7240',
279000: '41c48f5965e8939d1df69127798b46af17646ea75e545a7b47049596d9bfb3a2',
280000: '48da55fb8ce259432417ffacba5e064df4df2a8293cdd484ee0ad8ce4b0bc38f',
281000: '0e892fd450a8ca018567e1a6e23bcdc5b4b872686f816a415991f59b3abd76f3',
282000: 'b90fb78edf920d0bfb4bce1f39327c6777b511fd05d715729d7c4cdbf449d59e',
283000: '80965fd3817b9509ddbe6a919aa73adb11af64cda5d8352ad2bd9b7824e95c42',
284000: '43447043aea93b162d6f615e8cb7d992d02c176a41b0518256f7e0d8fb3162b1',
285000: '33bc92c77ba88251822698a7bd02860ef9c12420e4c27f4020370acaab0a2f37',
286000: 'eb664123736754c19a47e2afbbc8645c598d7b70c3d42132c334cfeea8fd6bf3',
287000: '6af8f2fb3ea1f0fd0d6432599c3a681be7d384cdc553903bcaac8de2d472725f',
288000: 'b5fcfef1b5f6373bfc9322164160fe98fc7fb4f3117bf91c8fe61324db62723a',
289000: 'f6f1c6f8a2a2905b2ca18845b8bc771bad6e9cc32fd2c8e20a565cbd7781bac9',
290000: 'cded625fb7b923d7471ee28276b6983bb4fb4f6281ded9a8637b6974710feab5',
291000: '49af69a4855a36d00cfe2408f7d090fb5001913aa06615eca5af78ecb92b1fb9',
292000: '2dfe91f7f1870539e6b51ce58f9cd6415ea8fd50dad376d74941a9ff1bf0f8f4',
293000: 'b84f14b784cf6eca78d13f4b0db4714be5bcfa59b73974b7106a0206a8993767',
294000: '07e95fc3c141dc64ae192cd233f97ac16b1adc9dfbbd685475bf693930b6f604',
295000: 'c39cca37abe0b8839f8e797287385c73031496c632a064a89b9dc0730a3fffd2',
296000: 'e69afa519ac761328358f5b3ac8b4395d5aa4b955bf1b4c5f3bc0af49cbc3756',
297000: '20c93e2d1d9bce1c861eb16febe50e534acc3e2d142cd2d0767bfe076545d1f8',
298000: '75970b654ac884cbd1328b68b59963bfa5babacca4c7d48368a1d938feb8af6c',
299000: '05448c7f1230870e814858ac9a7858fc72a045dd9e0f1f14b8862d2fdd524b07',
300000: 'e254223f85c3ba65a0295a5538387c08ae94e6070f5c989d27349978fef077c7',
301000: '4a8bd2357b4ce05422d51e052018fb9f75ce3c07c53b5d1623ac19afff388e63',
302000: '18f4a960b810034f5ea02458bbcb9bf3d38d681555be4e40d132f867444306c8',
303000: '667cd927ba1e7ab162c17a711bb0ba65f838e543f46b093c86c14e39988da156',
304000: '55280e92a3c3b9ab9e1032490aff10c1a2fe9e96cf4f88e49b7a756c0491701e',
305000: '8af5b503f1819b23da7b3663314a15b10e39e072377d1b08518a373c85515c1a',
306000: '439796b2b4add7dd0fbe14bb3d04f3917e92de2afa07da2a384b389ad9228888',
307000: '07fd4cbacddb504e23880dc454fb70659386afa21dab3e862cbe7688218a43fb',
308000: '2734a4205be5f2c8ce5850bee8d4ad0829c208911bf4ce833349a43af2db8fe1',
309000: '261b84349406a913a4cb700fd218112920910c1c57922e289114c7b903fe2d9f',
310000: 'bc652e239edd3476df7bd1e0563595145fd4f2b37dc2ac3bc28ec089a7550812',
311000: 'fd7c1b32015a2418efc6e1a73470ea3205de05662c767b0d4198fcaf804c3b5a',
312000: '4f017d27b81da7eaf62760cb7fe6613a6ec28fdc18859b6e727b216240a86315',
313000: '2d09525e2364e7cd7fe45191fee11f1ce16a2fbd16b56e41f90a3c64ce2f944e',
314000: 'c05770490a47280c5b318709c264b2ab951a5bf8c4575cbf1b1e3dbf35323f45',
315000: 'ac5da2f5abcee0f92f29479c6355ddafd1f3b4b3cca6659117c17c9fc10fc743',
316000: '4496d1a0e4c22a2521595bd4fae60abdb9404f79135311606a9e6713be373271',
317000: '40ba7c87490677af9e8b033a19ee5ffe51ba192f5912f491738bb6485cfb76d4',
318000: 'f42045295ce30bd4ba9fd9c845bc1228949b4152517185d3dd0598d8bae38481',
319000: 'f488695e46b53e4734f4eb9fced93f067f037af7099d0f9510856a898a0c438d',
320000: '83e42370911c70bfa2ad89a82ecd02bbc9cb13d24c2a900a0030771898ed618b',
321000: '74eabfbb64eb89488b59393ed529b4ee97e0f88a7fc14f798f22715bcee3fd19',
322000: '54f42438cf96fe8efa0180a47350002b34ccfe82123a85a9a6dc6f8b0894f0a2',
323000: '5509357b42476732435a1d5a6a850e27f62cb3ca81219a94f08dded06dfcbbf8',
324000: '1cd477361dc4c72b4f157469b6f3616e060f5d058b44ba864225210432b35540',
325000: '060ae9c541eee90a80a11fe300583e47c0ad4d12093d0651310cc6c87ecf2b75',
326000: '13ce59868c947ecf2d6aa986501c9f8411a59b4e0b8b6113f4f45dff4074fe59',
327000: '16a1532ff983f5cdab6c2d465d58f0369665a926c9d877212d95f9387e8732ca',
328000: 'c9608a16fc43e7e94f04b3309c067b0a4f68a66705009dabb39898f41fdc862e',
329000: '029b02acbe0fc8610894f0b89d6bd274362195a443fb9e5e4d4485be7786d461',
330000: '01181b9529da87eb39510619285c08b05cfa2ca2c7a2fb3ecadba474c1db3aa5',
331000: '2371752c35da119184e53dcb593a193fb1c81d6edcff921599328a6142a57e3d',
332000: 'fc025a8f319725faa9903eca86935116a2aad8fec6193055d5d5f0f431ba84b2',
333000: 'b602651afd9988075887a3f48455d977357032185b7d482262efff9cc97a45a6',
334000: 'a9a2bfb6b0f8db298b5ba732f4a23242fa89e1c5038916ccfba3751f7dd1b4a0',
335000: 'eeb3e35b6ea94119bf058e6b02a2ec2a706089bc1c360ca28d1b4e2ce1b547ac',
336000: 'e834f7ac6bd8e5b46ecebda4997ae73bccf353fa024fbaba7ad4d1d70d08d14b',
337000: '229a7017b6d0ffa08374e297b07f12620bcdbaaf5977dd39f3f207ecb03c7adc',
338000: '83d86500e8b2ef46a0f1505e29f7401b123ea5ed5feacb49864ab1de284eb574',
339000: 'b08e9dcef3e38d631414b4d3a8d8b3dfa8a07d9e5829c515e7db2fec64c60827',
340000: 'aa36ce6494df6905f4516530ddca77d88341c0202bf052347a86e090c03ab0ad',
341000: '527b50125d5479117ca8bfe84fd5507b2312360908d4cafe8588d40c4ef5622d',
342000: 'd1397e8b0f847b6d46c409659e9e10f3182ad9bdbca563cc2598af37cdc080bf',
343000: '896576aef0e0dbdb2df8291f2832716029f69718ac79f355425d475a1b2b7b4a',
344000: 'cf4b3c75279b0f7d71cbba934228709ff03e3254fdd83a47fde66e83ff393eb3',
345000: 'e68c760edf370e758e815d9363ada456fa2568b06abf8626f28a32cc62944770',
346000: 'ab19fe52e780e45a716ed97e93084bb3f7916e0a4479f131c7f5f7a2e7d69d9c',
347000: 'bc77d338a10fd71daa6c68fa9df2fdccf44b6d9b1d7c6456bb281bf8547b622b',
348000: 'f84469ebb26540cb3de0ae03f1f47452f9a9bc6b7195c3b5260d48623507ed86',
349000: '5bafc3e6dbe4bf88257c4381cb4f2fea0a4cde1f272e1f1006600b507cb0f670',
350000: 'e53ff96761b9c98bed635457acd1856a2ffe65b84e339cea94ad30f13a31e5e9',
351000: '5e0eb50d32e90e5ca73639313e5d3a850a113fbe60ac494ca6cdb799e9604d7e',
352000: '1a1a04aaef110493bf8fbdb8d1784a7706c6e303773ad0bad6511dd3c1da6c54',
353000: 'd72f3b615e244584f131f2f8e7154534aa9d2d9c4a710bb4de44de556b5d1232',
354000: 'e81b823c64229c555a50e90a4c3c4654b6673cbff4c62c6e897d09aac00d95f8',
355000: '11eed14cac2024588b389ddcd54315923ffff03d6c4c5f32657ba551fd308516',
356000: '465f28c63e564296f75a4a22c3ad16714cda29d7a2797e51e512eb1ed5f6a49e',
357000: '2b544c79c6f74ae1c05c268669e57ad41d6807f1d0ce1a5024b7b08646df6861',
358000: '8a76eccf113819aad8d7c17a459a614274bfdaeacfb4659d945f13dbf3907ccc',
359000: 'e63743f458f1479593d73952bfb882c72e5889331add576a716761eff2cf13dd',
360000: '0ab0bef439debf08dc3e1df9010c4e119d6862de6620de5f725cac0013c06301',
361000: '50f649f0c8bc0d77ffddd53fd5b896705a8858d3684a3ec876cb320ad26d2c29',
362000: 'e058c43c366c9c307ca3ce5a732f0ab2b4e09ff291a067366af25a82fb3cddf1',
363000: 'f99d7f0ec93ec7fd74273bb6f8bc4ad8ec0fcda9f6406001668dcf62a088a867',
364000: 'e5a69ea8763650542b1f90d6217af00da137a9bb72cd0dbd5379570d1fa696f0',
365000: '5d0eb3063474c6bbf9d9581898fcf8c4cd097fe5c6e3875d7cd701a5a0302507',
366000: '14da2978f4d4eeff5eb0a6bb663e514095acfe2210039df8dda9b76f522f74ba',
367000: '4ad3ebe23fe9b0f407c0765adb69ee973de5b84db29c97167c7a358fe8696bfa',
368000: '22db57ccf8c8193ae9fc08abe11f1ea8e869bfd6ab2c0b918820acdaafc19136',
369000: '530052d61878d20282a69e0bace41857f7edea6d3e9e92fd0a7781c8b60d208e',
370000: 'cd057d0c889c78e06d0eb71287c46deea4340b8c7a4c470dd69020e1a9029c43',
371000: '41f32ab47863f0c44b8755ecf9b201aaa92999051ccbf8a5acdee0e12dee4110',
372000: '37de29bd97d75a41dc3eb52bed9c09baf069c23120a393888e15e61ac7f81ea2',
373000: 'd6b26bb791fdf6d79f59d5c37ef2af0a35149737029157724224d614477527f4',
374000: 'de18c6a921838385950b10ed9429579e95a0bdc574f448f457ee486a59a04260',
375000: '1f86edf4ae47f409f8de6a16c17c2da063937478a0ccc800633e4d33b531709f',
376000: 'a0511050ef086052fe5671d3333c255507b03edbd365cd7ce7cecddb4000f391',
377000: '7b8f0060f51fd170296b9e000bb2970097f77ebf0b5872a9ac01be5aa03d7ffa',
378000: 'fc5669d194d822c55e141ddc9aa229059e4663169456dc927d43b44a2c8d3c0d',
379000: '1c0bf45f052e0123d1a6806fc4a93ac8261ed26772f324c1b3e7c3fa44d7f442',
380000: '73b69dcfa0a916a4c495a14d4d43e270b123ba37d267e3eb0d65890336fd3e32',
381000: 'e89e3d548193df667c637f774c76f508ec79b7bf1f2ac5d9adf7468006e34454',
382000: '6b835973c2efcd564fbd58d96226cb984d5c32dbf63d5b6044a0a11f77c2ccc7',
383000: '50b7011656c5903941303c7a19e4ea0fc6bc10456ba85638b0919658556e47cb',
384000: '74c509b238c1418ae527d370e8fe922f7639c88ef8d78fc3a45894deeb2252a2',
385000: '6f9f6147b06d8fea8c440cdebb3869b7bface7cb2355d66639060056adc9bcc2',
386000: '5b59e68fd62751b8be184a5621f1c63f1e9cbd22051487999816df392dca7837',
387000: '86d8b9628b83ac63637c86c364001067ee1c3050f395758fd3f0bcab20cef703',
388000: '5e570f7b69738285848bd8b7f2fbb82d05e99a72cd4a0faf20a9e3491d2d041c',
389000: '78d31bda70842388c1cfb2b83e9e95ded37a0304aaed3292e2687a96c07be9b4',
390000: '316da5e2ce79bbc4f4500ea6417271f0ae268f85f86f81223d0af38d1e0e0a67',
391000: '45cebf8a25c25aaaac06e915fb3fcc99a84be61a266c2bd8997bd1fe00a7321c',
392000: 'c5ff6e24f524bfb1f629245345052c0a0df33ab881ee9d11dcaf64915b2acad3',
393000: 'a54b5d4824bae06f91566b73c99a07b812095d7c133d1042ded80211c0402b3f',
394000: '0ae76082f8347ddf838ce0ad486f5decf6223e8b8cfc16dfe178b15830807c60',
395000: 'dc436c97ee6ed427687e6d48e609d271b125495afdebbcf611c0fd5102ec900e',
396000: 'a87871c13ca3119f309a4ed98e4224d4eff01387ea67e86148007ac0d759f2b9',
397000: '58c143b302132ad4b38718ebe1c5fa81c1177a30045f1e4a52dfa749e84dde1d',
398000: '33089400401a190c9375abb63c6ef489123f7369e5a35766b59e69edaf4e3b93',
399000: '656305fae087b86758bc7a01e13051b1aaf4a5c53d9b71e56614217a82b20eea',
400000: '7ea274642eac11d02aef589428b181e1c8aed913e248b357917604b00b23c752',
401000: '1969a9f64e316e0b4fc864fd7162379821510a944652b8dea747e425c6222807',
402000: '3700e1703e7803a4577de7e364a34c3be94d3f933e05c5dfbf6ff68349b6fa31',
403000: '8b622599bf4b2d82bc11c897f35536036e6798ac101a6256e14477624147efe2',
404000: '5f6f6dcea393ddccdec69b30881ab08e01e1a7e6a50d681883c759b742c12d6e',
405000: 'c3fb0507a57da69042e246dc8ecac101f33cf2b1fb53fe2f507ce28a0d82e4b5',
406000: '4f034663bb7b84439fa256f7b4ee2a5cf93e0048c1eb476aa2152453ee7be544',
407000: 'ff99ff38df6f928242571fed5b03fe0c3cb481b510d3c943e47e26f26440cbda',
408000: '89865af5e99e64cef61dd931061bbca7fe355218668f487f06474e46ebf892d8',
409000: '1b1d050c0b98d55d9ecb0d1a5b03e75caa352b4e0ce6a57136551000e150c480',
410000: 'fe52790595db35c7685e5261de78d139c830af0a5ae73a7e33da21be9ffcc9e6',
411000: '1afe55e66e6e16abff282c9d6bcfbc6a3a6e40db3c60323cd44e3b3a5d4ba924',
412000: 'b59a4057d73e8037f3e0e9b92310def7900c19917892ebe7689d5f8d77912388',
413000: '603f3ca16a6a795d5aa807eac5d6ee7f80a1e2596cd91cf41cf90ee309b2871a',
414000: '10e255807a43bf5b95f9af648f1724de1e953a5540819822be5de184ead24ae0',
415000: '9faf6b1c33006e3da3ae2c7599c7fdeb5078a8427edd6666290e32aea67f5a23',
416000: '973b77e4d491ce0c163ca64f5419b05822eebc2f5d85810f51370f56e0d812b6',
417000: 'bf99710d77fe84ae3ef5a371887cacc2ca1fbc5549b04ef9f61cb9d911b06a9f',
418000: 'f032acfc2d629fecfa180067b4531f9d72924c5f204cfe9053c6556a334373e8',
419000: 'bc9f085612af6a21411fcf51259387d97e6ae1169e4c6f92766a168302a98bac',
420000: '9136bd1fa7e8850c52e3cda39a96f754cdc605a69ea49b7d6d959d0cce81efba',
421000: 'ec2a78bad1b4b7325d574122ad8c3072d27fe6854a6965e6917a28e0c868a229',
422000: '536f4cc5d6ab082d32ccdba1caccb5d4f3145b60ee8538b1e18c6231dcfa23df',
423000: 'ac7ad1f97da0e55661bb25d53c27ea72f705d54050f020aad9fd5856a27117cd',
424000: 'f840eed1c0fdfba43965bfc175e4d92a5ddd0fd6022dfef988d70a5e7f363116',
425000: 'c1a8f9b79a34b76c9ae58e005501dbb1893cfbda8bb25b0d54854657b3ff1a54',
426000: 'e8895d8f797026a2f19ade2b548bd2cb655e2d756dcc0a50850fd37148d24ae5',
427000: '0ca56cf4fd882583854fca00e339534346d53178c6679f8eb1e7a5dbb7b9e7b0',
428000: 'd3ce901843f952df4b63e755a0f58924ae11324aad47afece9a93502dcae7c13',
429000: '774cf2d1ba62f20f5e1ce50692f65f0ce35c7456d3988cfbb2e1b753e0ab0f51',
430000: 'a07fc310f6201c30f1fa70a082348942c65743d51e3c19492596f6aab4bc9bb5',
431000: '7bc7f405bdfc91dd01877ccfd6142c973d81dab12fa188f468abe6c9a30fe0d8',
432000: '7fe40e145078c1952f87547156e2147144449971f108164d27bcfb93f1c840d7',
433000: 'dbe52d2772d6c0461e53357b71a79096140039ef643876ec2d27e1bdfee7e81f',
434000: '94b379e842d4979bbec171bb08881227ac4e9477186ab63f7eee9b4261b92365',
435000: 'bc9cd265de563cfe80090ce12085539bc67d406c23e4d41751f191d6b6c63168',
436000: '3ff03098d2c20b5cc080363a84d8295510db937272801454d429c4d17a1dbe10',
437000: '5b8ebd40170e217eb66990af3574b80f4e66bedd092e44836aa5101ded50c13d',
438000: 'f725813181d50c0b20552635ecc578685ab0218619d53ed23493c46fb060ecec',
439000: 'cc7cb296ae79850ba2f6f7532c445189452831957f62cb823bc83aa6b51daee9',
440000: '757b816392227c9d1a14f7350cab6e77a46e465e7853b1291abdaebcee2bba94',
441000: '834515861eb72aa472843a6924fa9d6350630ed56e35181d9fe66e3a46b6952a',
442000: 'cac31ac937ebb6f0cf4f3c33f19d66a05bdc11105a98ef12c32b494e19e772a0',
443000: '552b03680b5495c7bd6ec58db0b6767e6bde5c5961a02b7b2130ea6116bddbad',
444000: '2bea6ef419d9fe81de8bf5cfe21b2603f202e2b9a959d2cfbdcbccbb7a7406c8',
445000: 'a4106f2115b356eca92e5e3f45b8ae0e13b82af94bae391942200d1c59d34d17',
446000: 'ba49924afd8a40260e546e5e4633e945877ac9d5754c277b77bb154c0c9d0e40',
447000: '83ea56135c2b052df0fb09c68cb7a085d876854f34338abaf3cf51760c83c3eb',
448000: '806d77789673fa09d24c188ee024e71425df7baf623fda073076f3a9b87ddbce',
449000: 'c41d60b93fca1d2adbbfcfa99475a8eda152630d815eb1f6c58cbd15a0c2eb65',
450000: '732d315feeb485fcc5055916df17827d80e7f831c43df81ea79123882c4fb356',
451000: '202156d29e57c51941cdb7256918128589f5b8dbae5762bbbeacff6eefbbfc71',
452000: '0194e8482252f2492e1e2696fa5761f50875d0188310a0cf1d967edffb1b79f5',
453000: '9e63570821d8b5cd7de0c5f9fda67300f531d87cd8f778fdc6bee9519b77e01e',
454000: '632353fe048e83ebef942d64d550721d29b3fc8083a247b2443f2aa2b706c6fb',
455000: '65863feebdbd60d16330000cbae496667a67dd5fa8b8e3b6dd3732843f6222a5',
456000: 'bbc554cb3c0281d6853ff2f6c25b1251eb108c09e5a6c4a69a56e8f79c9206e4',
457000: 'fe6fdabb84ad1ffa807fd55db29dc95dafeb0e9a5eeb5ce75be8a4d16f9f116c',
458000: '949bd40d861e4b297352722b6f01efe9574711fe64b10c2e8dd769ad99cf9655',
459000: '9c139c9ccb41fbb3ecee878cf47bb37e28c76816c257f99dd7b23ec1f3eb3ab8',
460000: '61c3126d504fb38ea21948d2837368441389155e9675c7f1629eefcd03198f8a',
461000: '28ff145fe2abc7cacfb1613d40b45e13580c643ad68c44db81d15125324f4f33',
462000: 'e6b532924d07f1ff3976a53397327ccb232d3b2ac6ab090874cfa6ae47b8b315',
463000: '15ca651db9b159978ba6b2adfae1669e3042f4311ed683c72838bf337d71274d',
464000: '6903539d4f38f728e075441efa96b8013295ac166db50ddd4103fb2bc9937f0f',
465000: '6b6c2640411d900c13f5b53a9a993d10fbbd7f03f6fd477e120ba06a7cd67778',
466000: 'a05d282fcb9c091a0a96c2ddb3728c1e98eb767fd5311bcc08c9c10b5a0eb24f',
467000: 'afb81fdfd562f2cd2d53434e6458c38462f85b65958ecade001bcf2b49f9b28a',
468000: '97374586191b3a30654483722f7c6486ef9d9f71aaca44b72146c11d4f4b80c7',
469000: '90a829ff58be9ad30024ba1851515cd96d567be1d99154cb966067316d1902af',
470000: 'e9fbf2d2291e1b5af5dded3c223611d9f2e31fa6e1d8c075741aa155538bbddc',
471000: '3b4513a6a6c05ae9d2ab880a2d1aed3b189e0fca9cfa2bbc7c31ebcbffaed9a4',
472000: '32a57820226f672e695823620fccd42a6fe624dd2d88f7e7c76e0e9128883b27',
473000: '15cb7e920799939eb01d06b1b713cba3ed31ef92ca037ee98eb8101b6331e5a6',
474000: 'cc23e76e68dd726f31e592a4ea5d6aa3cacd827d068317a0a476f16ce468d9f9',
475000: '9e7e33a8ee53a4bd94a26b35bb43054b42a238a85a859ed9e4a33db978a128ff',
476000: '23c85e4d8edc7939fe0e5a47fa250f0921b366711c34910c394af00a4efd90c6',
477000: 'fd9d489fba4175c1fb3b8a0f9bb947287e64da4bee9f8a7fe73411632c820368',
478000: 'e287b69696013e70b276843e83af6cb5cd076000cb280e43de29cad7f706dec9',
479000: 'e86c7a06d96887e404c052b0fcedf10d40423b02d6b9fa7ccb9827551e5d089f',
480000: '74e7ad7d78dc585a5cd1054c0288aefcec0e031cd40f0ebaa0fefc7a325f212a',
481000: 'a01f98758cd3e561d02bef30c054ef70405ebdce9e670a7d6165efe3a9030886',
482000: 'cc9e22f04e4e2e4f52b8382281092ba50ec99722c9172a0e1c9ad654d26bd4d9',
483000: 'c92e278d54111ab2f3906eafab87fdb932307e60836df209856a97cb9e1ae3a8',
484000: '18fc89d07631aa01891aa44231dae6cfa43fd4733c193468b033532a4574eb80',
485000: 'ab546bb694363e07469dec48b559479d34d724a43865fa8f3a178ddc7b83e8a2',
486000: '59926fddc89abcfb55775a58e5e6e453e388f530f720dc61001f1ac59b59cd43',
487000: 'b41bcbd91984a3dfe72c87bb7c2ef32ba3eb3cf4318f5d37654459143dceca1b',
488000: '94e859b8f01ccbe9fc5bea5c274f252e23599cd444d5eb0962ece97110b86eda',
489000: 'fd99b951714162e55d8944e8c3ce9b098c4b74ea24017e4b204f95359202df08',
490000: '80b84d18f4bc6d624d4190db94920976926bbe48f2e3e0f4c08106b9f99ac5df',
491000: 'b0c5b14b01225b1ed8272dbe9b2def41756cda0a3fe6a1a8e75b0795b72e6a7b',
492000: '572dfe57507384278def462488416248a3039c1a76a8c4aa23d1ec706de1e3f6',
493000: '75f22a8d4d202914c0f3bffaf95e47eeb82eb00dcd2913af5dd67acf6f8ed8f0',
494000: '626e26524f9427582d61f340ee9181d8f238ad8acf96c1862724fab62ede8c8b',
495000: '69a11346ba65c79b5fd566768a8b2880a5a29e2552ee8e343bc88bcf43e9ca54',
496000: '314c88db22753d317c7b65f872d54d3ff672d04b3781293d59a1ea902811336e',
497000: '7bfcf2614b54ac3808894694e8b15940d6740760a2252ba4b47b2926bdd4bf51',
498000: 'cc02f90d2fd4e0f378d2ae3bf1011026ca3d7f6642dab316f5fbc2f57aa840d0',
499000: '841182d8519cd7b2d8fd01ddd087de892ecfafaacf9d6aa029ce1ed71ee0d538',
500000: 'e2742854018ecbfef71d626ab797b7c6137e84ff1e27a05b5da84b2fc7c43145',
501000: '07334852d0b542f897164e4d74e699678c850af4e46111d495701a312be21105',
502000: '6150771ac25f4185eb94ef6aa15e7f409b6beb411358ba7d982755655d431492',
503000: '156d3165eb11d021c46ae5968aeac20f65eb05e68d841bdc98dde156d42fb500',
504000: 'e9d474e59bf527d84f031ba41cf471f49e5bf0472ae1b3f48f036813a0b195e6',
505000: '261cda40eed5ec5b2a44edcd995b24887a45a417fec3f68507a4d600ef2abc33',
506000: 'ffbeca24a8f2b7cd6d64207911e153a2f0bb70d34bb4f073c33197e3a2276558',
507000: '14e2664bed708d57478f6fb5a4228e4c26843bd6533056fb38dbf1e7595028a9',
508000: '2750e25c3dde233447266f7b35f78d8bff18bbde4ae094d06e38fbf6cd2c6902',
509000: '5fc210a72d142dd413bc3b76d5ac772564a5917aef13890e7cee190c61b059d0',
510000: '7184fa5628bfe59481bb3542bb4212b535c3ede3c24425f0bf7b1b453c238c6a',
511000: '2b4bec04d73f0e524cf12d04bbf7b3859b489275edd047f41b139ee6220074de',
512000: 'ebf81767f4d26c98bce62a5f5d1a289da5bba770e88f346e2fbb1d676c155e09',
513000: '8bd8a99c65a258ebec809ca76e2cc91b0b4fec5dfe0970736d2fcd706aa660ff',
514000: '2dd7cba17366c483819ea090d3196950ee35a4bd9f28acd94575525c851234d5',
515000: '33ccf5f5230b9a79cd89eace91cd1c2858f6d040a11494ffb8d51ee934753d57',
516000: '30eafb000d199c8d9d1a55c5b09a382a3b10bcbca837e584db6647699ce6a3c0',
517000: 'e7ed622512e2483fb8537579ce2d2519053f9a6c8d19c42d665007632d5f74d3',
518000: '5180d11f98a25b8698b969c3c9a4b736522397859e7dc00665658f93c6cb8be4',
519000: '17ca81b3a3c942c80384c4d2c176b5274c52569b90eef686e077f53a884130c7',
520000: 'fe1f552778cd5250ba6556712217b86fd6f1d574143f8ad6238e1d6b67481f97',
521000: 'd7ac10bcb8662ab9940c4212dbf0c4b9cf9750e3c04a905efdffbce63f065de7',
522000: 'fa7625300aa890ccda58268eaf2daef14539c195f93311099754f955f367413f',
523000: 'a55ef67a0e75bea12003609359443a28b42b9037290872d6b1672433cf5c0d34',
524000: '08d1d556393bbddf6eda11df852a792515ab9c1d519bd91c48711a0de8faffcc',
525000: '76b8691b16f7dafcf980ec3e7c69c2f3caee32057ac8d61b3d3aef0fd394642c',
526000: '6d3ecbd7132a725e330f0a566d8d602a9cd96326a36387c1c1fe52f7f6ad975a',
527000: '71d08ce7aec6ff50fa1ddc37f6b1574db6e71004e8e111f04a9b715103f68e01',
528000: 'e9b103c11db50a121c9e3a1fb4cbbd4fb124b07a67167c5a461a539dac3f932c',
529000: 'd2f784907a019c1eb5830a74799562cd6d21570c4591bb038f379897f3296d75',
530000: '996bca5a7dfc4a8c7f9f8dfb38b1881fb97893a950f4c8f57462506782ee8ca7',
531000: 'c84dea2e7fb0c9c884f8dce525b8b65bb7deb33ab7fdbcf72a5382e9bac2ade5',
532000: '3e200d76a92dd1f6fcc420d44ebe446829fa5e0162177acd75a13c98f1477cd1',
533000: '37fa7cd81c658ada13d04dcf0e674017c6b7430bb0cd5a080b1b924396377b52',
534000: '9483c344293c376d2bd247f2846a5ff5038b27af48ea5badcde101fcd0113dfd',
535000: 'c9d4f9982955c07960da1e4f92803ba135f4584416e0767a3567e4d2a21e62dc',
536000: 'bc9abe4e4be06aa8f61b599d64923ba88d36886b638f413b12bf87ae852e4010',
537000: 'df8307b0097b4882740c18432ab2d7e64486cb7bb70b503923bd630ea277e9e3',
538000: '6e2accc5e65a85e0aad098dbc7021bba3cac37c70f6a0ff3135f86b90954bd2c',
539000: 'b7d9f50727cb512dbaa1ba073d227805e7adc70f9aee46b9721a65ca3d34a8fb',
540000: '19c695a6b8bd256fddfd00c53e6401829ae59ebdc1e71ac234f82ffb2991ed0f',
541000: 'cc07c3740f96ec8929c7c8714892a19815fe64f4bcdc4d1bf12cd0a9e1e82125',
542000: '23ffec3127ec3e093ec8221fa760510e63fcab31f9f0fb2b48727649e4308980',
543000: '6c73b242bb05af125ce4ea324d921ece54c9510980128605aec23cc465dbeb00',
544000: '51f215e5f11ae31853c4caaae1d01aac2dbafd00302b70e7198e213d485d5bfe',
545000: '2c36150f62c86d050e4c64d83aa0071ef4af0bf8e1e17623c740ea4ee7f3275e',
546000: '322e6d77f529412c4b7a1c7778253900f1976f238f96b771845b56aa080f4d23',
547000: '1d3413530079389231bdba0a0a7de7c85a4ca991569b2f26a1e48ad2f34a995f',
548000: '3576ec0d3ed469c5498229af531fdda290777fa1ab7cd83c49450f730c5ac0d6',
549000: 'd2a5ed3d920f273bf891ac40ece534a56f8774c8809ed7d8482ce4b78bdf016e',
550000: 'e1f75701ff829339e03c019eef2da1ee4239f38879173a833f81295200e96d29',
551000: 'fe8339fb53a33e3b5c4a39cdd64d869f41b4fc88c73fa018eb5245d2a7a01e94',
552000: '8c2180cc834d5c14c9801772caa65e82b44cde33da7aac586c81b2bfe77412dc',
553000: 'db9b4ea2cbdb21fd48a69da3bac2dbe8cea2a16d45f9252776633661e28fd1d2',
554000: 'd09c32f70dd94d737205922618b789204a39668b59fd3ef405ba6d4ba3bd8aea',
555000: '6e8efd0190d866972b469cc827330f84f515d6466d8748fc33520e8e0505bf3e',
556000: '8ce7cc8aa6ad30971014b332c64ce87303098e64391a9be03d1c621df851c00c',
557000: '9959211d82372edae69cfac47cc5c937fecbb6de9e235a5a3954c653f3b1fda6',
558000: '4f1d620babaa28a03d02c2ba8b7d1526ae47d45bac1220a1a794cf4a489536be',
559000: '3a0f1617c6056a27b1da33f0e4ca80b5b6648341551c19622f7f4b4f016ce8ce',
560000: '54877adf522c9abaec851a358ec6fba9957227e0e306d3a425167685380cf6ff',
561000: '55db8f8ad8869221ee891244767a7698d48ec61264729cae19008779dadba422',
562000: '4ac9c41b29253ed03c45222412c53ddde7b9da63441d06ea91b448d89dc220bb',
563000: 'cce11479fe1fcda6af734a03be8ea044df7e2726df67927f5bc76c33728133a2',
564000: 'e2e5c52e028ab1be7f5d182a217e16694ffd370930cdafa9443b627e8da256c5',
565000: '81488a98ee93b0cf5704cb1f46383456bf5238fdae7bcde4d0826276b5a48235',
566000: 'c9afdc804f7b0df600c51b9d596d6107a0cb71a4f8ed3ea764585ffc5f70644d',
567000: 'efd3ac4baf1c6e1fb2f8bac40f1416f189c9eb1135330ae4c25f296f012907fa',
568000: '225c3db3b83b78fbf9459c7e1e53ae35b01752608d329eefa99247d8a2b9b83e',
569000: 'ac98f3fb9084261c19a63fc8f10baeb5b03aaf9ca432d7f763a3a6706f81eefc',
570000: '4ee2eae15c9019466bb50d103947bdf60a5397f8fff56154ef782fd1b8579dd9',
571000: 'a89f29bfc76988560ccd59a61f8a323fde7c4020abafd577f88d1b0b4c289260',
572000: 'eb8380e096c79806443535f2f3b007378e43d2cf3ff56f3b7e6a1b2639ff3da2',
573000: '3182ec1555aae0df8d76dd5f37bebac6936e0e0c5609263263eef2e6e87863ef',
574000: 'fc45bbadf22417d49e7df89f02df358b85a4cc07adb4010d3bc0334f397b01d6',
575000: '6faadf2b0c3a98952bb047a58b83466a223dd323a6e32d68be5e8d064482fa84',
576000: '68465ee1d64b65ca71bbeeb611a8b4657d3ce765857d049536047e2ea0490044',
577000: 'a019fabcf35e7d4fe15e4e3a499dea2bedc61e4c9cea2803731995def15a9794',
578000: 'cc2fb6111fa1a338eeade31574b77a1012ec5948f88e7e5276f7ba4443df3db0',
579000: 'c5e7c605c03e56f221397f55bcd164dff21856f7fa48e745e61f2d05d7de5f54',
580000: '110fe7150988c930f9c324e00e108050b8be5ca2eaf200f54477073c72feef2a',
581000: '0c3adb09dca9a9924fcfe30f16bc70734ec62fd4a306ce65a38b74c5d817ee40',
582000: 'fc47433896c47765651d8cb31c88cbf80d0887bf071eaab9d2963af9f8bdfb56',
583000: 'c7c518f8634bc7c5cc536c9b77fe119cd6cdce4fafb25d4c0aba53c5bd970e35',
584000: '43b2357dd053608b30488aca98f5d9c151a68730fb8d008f1bd1488bfab0c391',
585000: 'cdfdb4ebd1c5fc3689b4f013254c832da7bfb12f3440fd327c6b065afd093dce',
586000: '48979b814cb6964b20ad8b322c4936c7b241c24fe6d610181c43d1cd003e5d90',
587000: 'c262a418368aab465facb8f1a82a392eb74e3fd56760779033a97a3f930630ff',
588000: 'b20dc4e0c17a223408a51b958b6b481ddeadad513e046c86c7418a8e96e35992',
589000: '0411c5ce49578218dc8e225c11276ed9a9802aac47d8f5837c4eadb3bd72e1dd',
590000: 'be8fe8b058722f4919af2834c076e2bfb602654a0343b57eb568d0ed4833fb39',
591000: 'affcbbd2a9fa43a64332412d649c2c0bd4b85e90e5e67576f09ee0c207a66cdc',
592000: '14a2d414220d240286ec6eddab0b0540ff4d1794b39f7913c00ab005e2ca2a3d',
593000: 'a1e2744404626b5e8e101402f50eb751c282ce692d58493ac6fa2efa43254526',
594000: '679da754e8f4cfd017ed5c8111361c0f9624104a87a50d077dd7bb5bb97c7ddb',
595000: '0c31eeb25d718f96c68de13704a0904c3a524b5936690ddab8b4b38fbabad5f5',
596000: 'f930b6516bb01d3365bd0e28aed56a580b4efbb45b0abed5ba941f7a56b545de',
597000: '56287fa7bcb53be966ba75c15b92ddf94a7bb51f4ef51037e56ccca936df4d41',
598000: '1a6303e68038921fe23644dd10f5cbad4af3648f7cbe8432860f439c84fb226b',
599000: 'add270bb4fcb8dc9c254c9130fa037dcedb8ca922c2e35ed403661a896addf5e',
600000: '786ea387bc7e1665655f1f683d56c67baeb9bc482bbe4401acd167793d34a508',
601000: 'a4c5f398f28cfcdc6e0b6fd467582ff5abfcb4f003cc59ae7703e9467cdd9ee4',
602000: '21b19a5a23344325581e7daf0a398b37cac5b4c335237bb8e6c14e270bdc1567',
603000: '000eef5b4300cfd804554e199ca6ffcfe73ec8446e82ad32abbc51f45594a368',
604000: 'fbf77437dfba9caa4d11b03a3673f2274dbd3d92e704bc5e8ed3f90d99d2c988',
605000: '0bdba251f44baa12e2b33b1ba89eb8a5d31ecaf6193b836e6356845149ea3f33',
606000: '2dcadcae1cdc68ab6404606937aa9981400e12ae77474462764b8077d391d2d4',
607000: 'eff9dfe7fb3e31536dc12c78603f023f8f1f59b2b44a59359e8bfe50b2e22299',
608000: '50fdd1223efaaa261f6fe592d7b9e1e32b5a50f78dfee7bc8af19250f8a82776',
609000: '5d88ff14c7c36602251061379179ed87c5f7d60d746d8d10fe678a0303ce6596',
610000: '5e56249989c623bd7f818ac1e64c5d1b1f3afec3678559d446e356ee5bb3394a',
611000: '5a9b15a38dd35b9ef0a028b12ab69b0993eb4bc5577d50c486fdfd032f243a14',
612000: '258213d8b7141418f22e303ec43005bae0508a3597fa99c4d59d656d7d768af2',
613000: '5ee8b46ebdec5576550540f7fe01c5a91c3ba7bfab26db605ade4e620430cc21',
614000: 'c41a592e2042d201bf028bbfef484d264a391db9dc9ef302708e7a39f723cf5c',
615000: '70fa6591c0b7bb63665f040b87750668be47058942c79c863bc00232f536d725',
616000: 'f4932c1c8fc84e3f4b97336424e27eeb09a65d7347097e983e1ab698a38dcc0a',
617000: 'd82bd9b58adf54a50bfd471c66144f2969ab443648fb7bab8299408afc021544',
618000: '72f7dfda9bd67b900106e632cf50204fe27cecdeea51d9b0cfde9ba6da8dcd5c',
619000: 'ac434b8d723ba498bee878cf589c7d61b74c7a00ea5b7abe70149ba878522332',
620000: '519930116d8a8f3b01ba443bbe4fe5346cebc783c632cdd5f62d305f61b6f446',
621000: 'fed977a0174f8cdc3e979e7435ffde2f7dbac2d4e1c63593117cfe81f320bfec',
622000: '1a6daef00425e6ad136d7bda36cad99baa762f6d788b92f630ed1af33995211e',
623000: 'b9f1e2836bd2acae3a9d21516052007005f3f67ae1fc133c1c4c336abe0dce51',
624000: '5d4a19f7e7f99b5c39bc844b0a90ebcbd97d88f817f8f2edafca78ae499c3167',
625000: 'c1afa80aebb856746da25503eb786f247ab098a94ddc58fc96ca46caee7019c3',
626000: '222c2ec50130a658ece226c420c76a8f7a15640033fd89ccaa29bd7c639cb94b',
627000: 'de9830e9cc7537c42133153cccf891dcd422203827b1377df748ec27dbfd04a9',
628000: '35d44ad84a3d9d548bbc8c91ca734979759e3a03e37153b650f794a0aed1b742',
629000: 'c119f3acbd761598673b48eaf0d223e1729b8c8aaef9a715f1bc3dde561a4568',
630000: 'ebfda8a47983ced56d40e95ce3be793305899cb92359ee2f9a83f7f6a59a4fe7',
631000: '1479c2e6217b97e98f44bbb315ca99d1ab02aad14f52d60f7e6670f5c6d0fbe4',
632000: '6ed7f9c90ea95825b1aae91644a68e36b6452e61f05a2821e94399823ad669e3',
633000: 'e00eb0bffcf784f9f8fef21b92b9cd98e44db5d790b6eab309d0b11adf910820',
634000: '3c71674dab2c2644d910acff4336fb02c2dd8281e30e5c97b9378d50f3d6eac0',
635000: 'ee3fbedd791f31de2f820cf37cf5a7bcf1fddff354c1ab5736384f41846894f5',
636000: '551b7598f5576c577b48ac7530e56422a5fd31a3860b37994f8489308830e9f6',
637000: '531b1a767734f1750077ec9d1ab57fd4430e0154deca2fdb9dc5cfcbe980726c',
638000: '3ae3a009b3822211b3bbe9745f006eaf3aeae365e81890205e0223925461af81',
639000: '2aa1de8c824fcc1a8f34269010405a0a0fb921fa0f8b998010e4076fd722948c',
640000: 'badc624e2ffabc438b7b2c9894e2134e36c2e80af700cebdf0980169c1cfe49a',
641000: '0b550e8f9b66ac3a3d0e5a911819643600dec121959fe2b5128cc6c68503e655',
642000: 'bfccbe9a0a127d4cf606adb5eb42f3f0ee2ebc780b719d0ec612e590aa7ec47f',
643000: '4d44b13390410caf86d145103471469d0b10450166f5b96b8886a440aab5b1cf',
644000: '2670950e311d8230e8d3447e97652cb5592248b59968bedb6a98a98c12b8d81d',
645000: '6703a25f54614ff79c954438ab9fae5baff32c81fa47306b337f0f87bb11f949',
646000: 'c93876f02c788041b400aa42149c176e624eef01bc66b49adf8fcbe255dd1833',
647000: '276521b16f6a98f5399a9754621fbd7999b6b94e3cff17cd88d4a1176f431353',
648000: '65b6238c589baec511c240cc87cb6ce758ad6cf119f465838a976a853c7f5be6',
649000: 'c94f6f7fa7a8afdbecc7c291eab2a3cf218ff1af5117ed1f3f0e60214ec7041d',
650000: 'b823008cdb0176fa0522c26b35962b4325fd526217f7eb19464cffcb3c24b4d9',
651000: '77024b0bfd3cc7c8d52eff6db4dcec041f8675e6f3c413b7d581bedb1b53bb11',
652000: '22a677eef4cfcee12756492eab7e184bda054481d4fc19cc46bc7147c097d118',
653000: '66779d92b9fc28fddc46ed14b79b26b3ad7672f0dbc7cf4462d5a49769a89470',
654000: '097eb77aede84c35822b3700673efd0129de247cb8ba5ef6b8e6dd7e85bb908b',
655000: '83b5b4d4aeef3782c82b9a2fd53937e4523ea15867780e4264688e860fd99593',
656000: '99646218016eda7976b45fbb6e891fc4bc130bfb3b5e6dcac8b11e0a4b2f59bd',
657000: '9ad1995af3f7f6839a11bf5fcd856f471b200911e1e5a647390a7aa26abc6825',
658000: 'fd5a4462737448dcfa9d81802d16f4621ea0b177f538144d836829a536cdd451',
659000: '7f6baf86cafbdc820437d63260fcc434694dcf0ab8000307f134fc9c50f437b6',
660000: '54a5f2c5a1f1534f23f54c5ac6e3f794ebb62f298dc9ed1aba4112e10cc778cb',
661000: '91b46c2d247a43ea916adad7edcc37e18d149fe3ab970eaf39cd060a9c856ce2',
662000: 'b49e0df59bfc9dcd32cd1a4b903a12accda70054fbd5f4bc480de0367f254c21',
663000: '29d2d3e06f744ec2225659199bd43e446308fe1e1ff16c26e0e89caa468450f1',
664000: '50b0665a8c7a9fd1ffb165fbb0148115f83fb7a5f6e9b8ad16c9dff9175d11c8',
665000: '35754b82653110c6cb0f9a0e7011ed1ff8b360301e4280e6aa2f6faa5950a369',
666000: 'fbc64efa658b00bbb70f5db8265e99465a07627d663c9a716ec8b42bed82338e',
667000: 'a046d6957f24ec9f66c73612d3eb483c65e4216fc2761260d9818486cbebe56e',
668000: '5d78ef63ea8ce0f66032368effcd56fa443b0f5c488ef8c243d3622113087fbe',
669000: '0b1b551500e2c8617150a18deec1e3b9594dd98fc9b97e20dacf2f059f966692',
670000: '5fa21e8edbdbb2ccccc2e3aca8e15b0965cd307ea54b6621760e654b53488d33',
671000: '90affcb451a786aa42ff93dd1f39961a15cc8c332613f8dc591d9f9c34d359e6',
672000: '1a8996b27fb79d43627caf0166f5c8ec72e1e0d09e6c0e17a0d9418d37719afc',
673000: '491ec5299fa141a82b438ef5d2429eb5560a0f04412878165c86f9fbb94198af',
674000: 'dd680d4c1101a98b6465ac85a63ef12f3ebb1eb88277d37aa341a5b387e2f1b6',
675000: 'af8acc98b0fd9ff347acb01801f246c63c6698aff0a0cd69b0f91e17da3742df',
676000: 'af24a5759888314e5ef719ee23b9f948b3962ac0b949231a540d45eedaa614f8',
677000: '7ec568d4e5ceff99c0e9cefa6407784aa0246845420812bcfd8a5d9c03cb01a8',
678000: '845df5d7e1d6f22242ffc4c976937da67ce0880a94c5bf189195d5728f208976',
679000: '41f895ce11bd91e09ea8b16102c8f2192cdd2c754e2cb04d33edea688fa7c3b8',
680000: 'cbf1e3a313abbe05d90ff1de6d861cf58da50b7f8f7ba43a593a3b6192d65f8a',
681000: '1ac184be2b5edfdcc0a73a6a93d3f59a5197bd5bba2ddb1ce737251526998bc9',
682000: '0e793523ba0a679dce1e9cf1d1189c5311e27ae8d88c35bce36ddb3c16dc34c4',
683000: '08e07172a6b6c3d70c4379b206ed796ea8d916d314d0b6f02539becbb90077d5',
684000: '0b67663fc37940b4d40ba88b8b7610776e35f070e4995c5ab09ccd4b86dc1143',
685000: '8ae92af68bcc012326889b5ef89b899aa38f65dfd3d9f7dd5b29dba0c5fbebbd',
686000: '5223ea0cffd3c59fa6597208425ab55b3a76366fbd27d22508cd208ebf2a2eec',
687000: 'e160e55af7fd110905364f980543fca62a123c84c81e9fd38a0aeeebc30a501a',
688000: 'fcf3b5e0afae8d665cd6f63dacd2a861aecac20f8b73a682f79cad9523c5c4f3',
689000: '772ffab484f07ad91ed45cf6f569eff7440f4243e16e07698e4b7bd4a109e6e3',
690000: '5b2a4fc69617ccf1787ee40a0f6d7e0af783bcd856c2c8e0ab747a91a7c68d19',
691000: 'e4ba352a759e2425d508a4d5b58595e6ff5eb912d8bfece5a0bc646f61e77084',
692000: 'fca2c5b6a721db85278ab55b5e2e39a445e5b1ee5ba69a17044623a6b945d0b2',
693000: '99f4365eee70f86499ec26c373922d389cbc5e2a198e96af5d5823ad241748a1',
694000: '0e19e76398e65cb01517dcd4ee702c9c04c0fd53cf9468ee539094aa6248e1c8',
695000: '5addd92022007d81ced43595458e2eee2903227063af8e9edd75cccbe559930d',
696000: 'ba25dfe3467cc9999eb8593d955032475b777cefc8006851f692266ebb83d140',
697000: 'e10c1c734038198a99bf970ae89e3a56b2058612e35eac8141d48e147d2d47e0',
698000: '472c4de8e57737ca48c3bdbf3c35c37f24a179f5dfa48af89efda3c3d33c131f',
699000: 'b359f2e61a3ca3cf8b613045617e38b06767ccff72129f13971faf26a4d08234',
700000: 'd0555d53358978ae0abcb09f911c1b3e7b5a282b4ff6d455ca9ad04299666dc5',
701000: '5c763b8b4329809553fb58ab279ebbc6639695c45760f0c181168d41da95e6d7',
702000: '8a6abdca484fbdac5fd6fb279a435377d964d43b62393b0d5b0cd3baaab1d2c3',
703000: '50cf159af75b28c7a95ca990e480ebcd534c4601b03d7c0a979f26f4e33cecd7',
704000: '0daf7f9c7eca5c6d91de9ebadb3ef1287c290700bab0aa9f6a2ac4bf42a8a98c',
705000: 'a359e7584112eadd7fad5d961f1dee180481852d7706701c2211a98a009c128d',
706000: '139ab9432b0c7f62739f818b114d57a59cbf20584230f1198c35f1da62c2ed62',
707000: '43e1f4c83d26d419a35adc4dc5bd85c6253ec0faf7e70196bc5d5fac18c51746',
708000: '1a05dba8d5d2ef4984c79cc939148ad71535043640fd5644141cdbc512623514',
709000: 'fe7a8bd03f742de5227e5d0c8dcfd0b5cc13582703b74b1898c506fc6ffdab04',
710000: '26217b4ecadd39e9a14ffb6cbcaab6ee7bf0954efcbbbec4de9bddb461819084',
711000: '94aee5cc00e862e4952e573908449f4a48b630285f2f2fb20765d01dfcb68ac6',
712000: '21fb256c0f5133634b94f6522cb8d2b0a9b982f1672ba561033be689410e7860',
713000: '8d7d7ee4cb0598fecd85c8004b59074bedc3219f2376966e01a7faa92377cdcf',
714000: '56a033531fc8dbb7ffe07d7365cce20281e522c3ff9fe7a19d0188af351a5799',
715000: '4a6671846cccdc26d6fd1d77db50772315f932c24dc706004e11de68ad1ac387',
716000: '4222447b25305bc063b668c0e010b16f9c0802fff9371e9d096ba2174926f073',
717000: 'c231fcf5238a34cd15ac735d27b3dd0ee025714e48c408a798f0ba74be0aef77',
718000: '0f96aaa30d20fc572d9db26871143a657b706c35a9668d31e3b64280a049787a',
719000: '8557c088c4fc4745674c795d4a58aeb3df0dc91bb5ff93cc6268c4a88c64db5a',
720000: 'd699e06ee7835c9d687853187125f27145ec91daf394fdee37218b8c34fee9f4',
721000: 'c675e891a36425627e20de36f1fbdd5baba7114661f1c45f81f66a3fc55da902',
722000: '0c6fedfb6d6c1a77254904fdd2400dedce7d45bdc2271beb77e2087a0ba30d1a',
723000: 'a442c320886beccb3d7ea13276dbef8e98e1a47686cba2cdbeb6a2d2883af928',
724000: '1592e2d6ac3be7535b44db6cc99080d00b19c6663f52eef3c28eae3dac27ba49',
725000: '45b2a800f17b8571172a2658577dbe95b91ae88e611a91ac0c92609b3600f693',
726000: 'ed6257a6567665747aa354e93ab7d3e6539d6dd41fced8a2f62cf848e2b30ce0',
727000: '4b1a577c6c2358b0344bb1befd9b8e5572b787ec2bdc0bdcd4a150f26b2e2ab7',
728000: '448765fbdf6261c376120ff9401db8a8841fcbed466365f982d3bc53775b93ca',
729000: '99c2acea0af193d2e10498acd1c6d162d2a804a69157af46817b5ece5ea86491',
730000: '94cec967e44f850f512d4240cb8a52ffaf953d0364b0a1dd7604b4a01406e669',
731000: '6e63f5019439bc7e27a17a189baad0da8f5724883af3ca35efa0d4e5aaa75b97',
732000: '53e1b373805f3236c7725415e872d5635b8679894c4fb630c62b6b75b4ec9d9c',
733000: '43e9ab6cf54fde5dcdc4c473af26b256435f4af4254d96fa728f2af9b078d630',
734000: 'a3ef7f9257d591c7dcc0f82346cb162a768ee5fe1228353ec485e69be1bf585f',
735000: '9bc81abb6c9294463d7fa12b9ceea4f929a5491cf4b6ff8e47e0a95b02c6d355',
736000: 'a3b391ecba546ebbbe6e05c5222beca269e5dce6e508028ea41725fef138b687',
737000: '0f2e4e43c76b3bf6fc6db9b87adb9a17a05e85110dcb923442746a00446e513a',
738000: 'aebdf15b23eb7a37600f67d45bf6586b1d5bff3d5f3459adc2f6211ab3dd0bcb',
739000: '3f5a894ac42f95f7d54ce25c42ea0baf1a05b2da0e9406978de0dc53484d8b04',
740000: '55debc22f995d844eafa0a90296c9f4f433e2b7f38456fff45dd3c66cef04e37',
}
@@ -1,6 +1,6 @@
import struct
import binascii
from torba.client.hash import double_sha256
from lbry.crypto.hash import double_sha256
class InvalidProofError(Exception):
@@ -50,12 +50,12 @@ def verify_proof(proof, root_hash, name):
raise InvalidProofError("did not find the alleged child")
if i == 0 and 'txhash' in proof and 'nOut' in proof and 'last takeover height' in proof:
if len(proof['txhash']) != 64:
raise InvalidProofError("txhash was invalid: {}".format(proof['txhash']))
raise InvalidProofError(f"txhash was invalid: {proof['txhash']}")
if not isinstance(proof['nOut'], int):
raise InvalidProofError("nOut was invalid: {}".format(proof['nOut']))
raise InvalidProofError(f"nOut was invalid: {proof['nOut']}")
if not isinstance(proof['last takeover height'], int):
raise InvalidProofError(
'last takeover height was invalid: {}'.format(proof['last takeover height']))
f"last takeover height was invalid: {proof['last takeover height']}")
to_hash += get_hash_for_outpoint(
binascii.unhexlify(proof['txhash'])[::-1],
proof['nOut'],
@@ -74,9 +74,10 @@ def verify_proof(proof, root_hash, name):
if 'txhash' in proof and 'nOut' in proof:
if not verified_value:
raise InvalidProofError("mismatch between proof claim and outcome")
target = reverse_computed_name[::-1].encode('ISO-8859-1').decode()
if 'txhash' in proof and 'nOut' in proof:
if name != reverse_computed_name[::-1]:
if name != target:
raise InvalidProofError("name did not match proof")
if not name.startswith(reverse_computed_name[::-1]):
if not name.startswith(target):
raise InvalidProofError("name fragment does not match proof")
return True
+245
View File
@@ -0,0 +1,245 @@
import os.path
import asyncio
import sqlite3
from typing import List, Optional
from concurrent.futures import ThreadPoolExecutor
from lbry.schema.url import normalize_name
from .bcd_data_stream import BCDataStream
FILES = [
'claims',
'block_index',
]
def make_short_url(r):
try:
# TODO: we describe it as normalized but the old SDK didnt do that
name = r["name"].decode().replace("\x00", "")
return f'{name}#{r["shortestID"] or r["claimID"][::-1].hex()[0]}'
except UnicodeDecodeError:
# print(f'failed making short url due to name parse error for claim_id: {r["claimID"][::-1].hex()}')
return "INVALID NAME"
class FindShortestID:
__slots__ = 'short_id', 'new_id'
def __init__(self):
self.short_id = ''
self.new_id = None
def step(self, other_id, new_id):
other_id = other_id[::-1].hex()
if self.new_id is None:
self.new_id = new_id[::-1].hex()
for i in range(len(self.new_id)):
if other_id[i] != self.new_id[i]:
if i > len(self.short_id)-1:
self.short_id = self.new_id[:i+1]
break
def finalize(self):
return self.short_id
class BlockchainDB:
def __init__(self, directory: str):
self.directory = directory
self.connection: Optional[sqlite3.Connection] = None
self.executor: Optional[ThreadPoolExecutor] = None
async def run_in_executor(self, *args):
return await asyncio.get_running_loop().run_in_executor(self.executor, *args)
def sync_open(self):
self.connection = sqlite3.connect(
os.path.join(self.directory, FILES[0]+'.sqlite'),
timeout=60.0 * 5
)
for file in FILES[1:]:
self.connection.execute(
f"ATTACH DATABASE '{os.path.join(self.directory, file+'.sqlite')}' AS {file}"
)
self.connection.create_aggregate("find_shortest_id", 2, FindShortestID)
self.connection.execute("CREATE INDEX IF NOT EXISTS claim_originalheight ON claim (originalheight);")
self.connection.execute("CREATE INDEX IF NOT EXISTS claim_updateheight ON claim (updateheight);")
self.connection.execute("create index IF NOT EXISTS support_blockheight on support (blockheight);")
self.connection.row_factory = sqlite3.Row
async def open(self):
assert self.executor is None, "Database is already open."
self.executor = ThreadPoolExecutor(max_workers=1)
return await self.run_in_executor(self.sync_open)
def sync_close(self):
self.connection.close()
self.connection = None
async def close(self):
if self.executor is not None:
if self.connection is not None:
await self.run_in_executor(self.sync_close)
self.executor.shutdown()
self.executor = None
async def commit(self):
await self.run_in_executor(self.connection.commit)
def sync_execute(self, sql: str, *args):
return self.connection.execute(sql, *args)
async def execute(self, sql: str, *args):
return await self.run_in_executor(self.sync_execute, sql, *args)
def sync_execute_fetchall(self, sql: str, *args) -> List[dict]:
return self.connection.execute(sql, *args).fetchall()
async def execute_fetchall(self, sql: str, *args) -> List[dict]:
return await self.run_in_executor(self.sync_execute_fetchall, sql, *args)
def sync_get_best_height(self) -> int:
sql = "SELECT MAX(height) FROM block_info"
return self.connection.execute(sql).fetchone()[0]
async def get_best_height(self) -> int:
return await self.run_in_executor(self.sync_get_best_height)
def sync_get_block_files(self, file_number: int = None, start_height: int = None) -> List[dict]:
sql = """
SELECT
file as file_number,
COUNT(hash) as blocks,
SUM(txcount) as txs,
MAX(height) as best_height,
MIN(height) as start_height
FROM block_info
WHERE status&1 AND status&4 AND NOT status&32 AND NOT status&64
"""
args = ()
if file_number is not None and start_height is not None:
sql += "AND file = ? AND height >= ?"
args = (file_number, start_height)
return [dict(r) for r in self.sync_execute_fetchall(sql + " GROUP BY file ORDER BY file ASC;", args)]
async def get_block_files(self, file_number: int = None, start_height: int = None) -> List[dict]:
return await self.run_in_executor(
self.sync_get_block_files, file_number, start_height
)
def sync_get_blocks_in_file(self, block_file: int, start_height=0) -> List[dict]:
return [dict(r) for r in self.sync_execute_fetchall(
"""
SELECT datapos as data_offset, height, hash as block_hash, txCount as txs
FROM block_info
WHERE file = ? AND height >= ?
AND status&1 AND status&4 AND NOT status&32 AND NOT status&64
ORDER BY datapos ASC;
""", (block_file, start_height)
)]
async def get_blocks_in_file(self, block_file: int, start_height=0) -> List[dict]:
return await self.run_in_executor(self.sync_get_blocks_in_file, block_file, start_height)
def sync_get_claim_support_txo_hashes(self, at_height: int) -> set:
return {
r['txID'] + BCDataStream.uint32.pack(r['txN'])
for r in self.connection.execute(
"""
SELECT txID, txN FROM claim WHERE updateHeight = ?
UNION
SELECT txID, txN FROM support WHERE blockHeight = ?
""", (at_height, at_height)
).fetchall()
}
def sync_get_takeover_count(self, start_height: int, end_height: int) -> int:
sql = """
SELECT COUNT(*) FROM claim WHERE name IN (
SELECT name FROM takeover
WHERE name IS NOT NULL AND height BETWEEN ? AND ?
)
""", (start_height, end_height)
return self.connection.execute(*sql).fetchone()[0]
async def get_takeover_count(self, start_height: int, end_height: int) -> int:
return await self.run_in_executor(self.sync_get_takeover_count, start_height, end_height)
def sync_get_takeovers(self, start_height: int, end_height: int) -> List[dict]:
sql = """
SELECT name, claimID, MAX(height) AS height FROM takeover
WHERE name IS NOT NULL AND height BETWEEN ? AND ?
GROUP BY name
""", (start_height, end_height)
return [{
'normalized': normalize_name(r['name'].decode()),
'claim_hash': r['claimID'],
'height': r['height']
} for r in self.sync_execute_fetchall(*sql)]
async def get_takeovers(self, start_height: int, end_height: int) -> List[dict]:
return await self.run_in_executor(self.sync_get_takeovers, start_height, end_height)
def sync_get_claim_metadata_count(self, start_height: int, end_height: int) -> int:
sql = "SELECT COUNT(*) FROM claim WHERE originalHeight BETWEEN ? AND ?"
return self.connection.execute(sql, (start_height, end_height)).fetchone()[0]
async def get_claim_metadata_count(self, start_height: int, end_height: int) -> int:
return await self.run_in_executor(self.sync_get_claim_metadata_count, start_height, end_height)
def sync_get_claim_metadata(self, claim_hashes) -> List[dict]:
sql = f"""
SELECT
name, claimID, activationHeight, expirationHeight, originalHeight,
(SELECT
CASE WHEN takeover.claimID = claim.claimID THEN takeover.height END
FROM takeover WHERE takeover.name = claim.nodename
ORDER BY height DESC LIMIT 1
) AS takeoverHeight,
(SELECT find_shortest_id(c.claimid, claim.claimid) FROM claim AS c
WHERE
c.nodename = claim.nodename AND
c.originalheight <= claim.originalheight AND
c.claimid != claim.claimid
) AS shortestID
FROM claim
WHERE claimID IN ({','.join(['?' for _ in claim_hashes])})
ORDER BY claimID
""", claim_hashes
return [{
"name": r["name"],
"claim_hash": r["claimID"],
"activation_height": r["activationHeight"],
"expiration_height": r["expirationHeight"],
"takeover_height": r["takeoverHeight"],
"creation_height": r["originalHeight"],
"short_url": make_short_url(r),
} for r in self.sync_execute_fetchall(*sql)]
async def get_claim_metadata(self, start_height: int, end_height: int) -> List[dict]:
return await self.run_in_executor(self.sync_get_claim_metadata, start_height, end_height)
def sync_get_support_metadata_count(self, start_height: int, end_height: int) -> int:
sql = "SELECT COUNT(*) FROM support WHERE blockHeight BETWEEN ? AND ?"
return self.connection.execute(sql, (start_height, end_height)).fetchone()[0]
async def get_support_metadata_count(self, start_height: int, end_height: int) -> int:
return await self.run_in_executor(self.sync_get_support_metadata_count, start_height, end_height)
def sync_get_support_metadata(self, start_height: int, end_height: int) -> List[dict]:
sql = """
SELECT name, txid, txn, activationHeight, expirationHeight
FROM support WHERE blockHeight BETWEEN ? AND ?
""", (start_height, end_height)
return [{
"name": r['name'],
"txo_hash_pk": r['txID'] + BCDataStream.uint32.pack(r['txN']),
"activation_height": r['activationHeight'],
"expiration_height": r['expirationHeight'],
} for r in self.sync_execute_fetchall(*sql)]
async def get_support_metadata(self, start_height: int, end_height: int) -> List[dict]:
return await self.run_in_executor(self.sync_get_support_metadata, start_height, end_height)
+58
View File
@@ -0,0 +1,58 @@
import re
import textwrap
from decimal import Decimal
from lbry.constants import COIN
def lbc_to_dewies(lbc: str) -> int:
try:
if not isinstance(lbc, str):
raise ValueError("{coins} must be a string")
result = re.search(r'^(\d{1,10})\.(\d{1,8})$', lbc)
if result is not None:
whole, fractional = result.groups()
return int(whole + fractional.ljust(8, "0"))
raise ValueError(f"'{lbc}' is not a valid coin decimal")
except ValueError:
raise ValueError(textwrap.dedent(
f"""
Decimal inputs require a value in the ones place and in the tenths place
separated by a period. The value provided, '{lbc}', is not of the correct
format.
The following are examples of valid decimal inputs:
1.0
0.001
2.34500
4534.4
2323434.0000
The following are NOT valid:
83
.456
123.
"""
))
def dewies_to_lbc(dewies) -> str:
coins = '{:.8f}'.format(dewies / COIN).rstrip('0')
if coins.endswith('.'):
return coins+'0'
else:
return coins
def dict_values_to_lbc(d):
lbc_dict = {}
for key, value in d.items():
if isinstance(value, (int, Decimal)):
lbc_dict[key] = dewies_to_lbc(value)
elif isinstance(value, dict):
lbc_dict[key] = dict_values_to_lbc(value)
else:
lbc_dict[key] = value
return lbc_dict
+63
View File
@@ -0,0 +1,63 @@
from binascii import hexlify, unhexlify
from lbry.constants import NULL_HASH32
class TXRef:
__slots__ = '_id', '_hash'
def __init__(self):
self._id = None
self._hash = None
@property
def id(self):
return self._id
@property
def hash(self):
return self._hash
@property
def height(self):
return -1
@property
def is_null(self):
return self.hash == NULL_HASH32
class TXRefImmutable(TXRef):
__slots__ = ('_height', '_timestamp')
def __init__(self):
super().__init__()
self._height = -1
self._timestamp = -1
@classmethod
def from_hash(cls, tx_hash: bytes, height: int, timestamp: int) -> 'TXRefImmutable':
ref = cls()
ref._hash = tx_hash
ref._id = hexlify(tx_hash[::-1]).decode()
ref._height = height
ref._timestamp = timestamp
return ref
@classmethod
def from_id(cls, tx_id: str, height: int, timestamp: int) -> 'TXRefImmutable':
ref = cls()
ref._id = tx_id
ref._hash = unhexlify(tx_id)[::-1]
ref._height = height
ref._timestamp = timestamp
return ref
@property
def height(self):
return self._height
@property
def timestamp(self):
return self._timestamp
+370
View File
@@ -0,0 +1,370 @@
import base64
import os
import struct
import asyncio
import logging
import zlib
from datetime import date
from concurrent.futures.thread import ThreadPoolExecutor
from io import BytesIO
from typing import Optional, Iterator, Tuple, Callable
from binascii import hexlify, unhexlify
from lbry.crypto.hash import sha512, double_sha256, ripemd160
from lbry.blockchain.util import ArithUint256
from .checkpoints import HASHES
log = logging.getLogger(__name__)
class InvalidHeader(Exception):
def __init__(self, height, message):
super().__init__(message)
self.message = message
self.height = height
class Headers:
header_size = 112
chunk_size = 10**16
max_target = 0x0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
genesis_hash = b'9c89283ba0f3227f6c03b70216b9f665f0118d5e0fa729cedf4fb34d6a34f463'
target_timespan = 150
checkpoints = HASHES
first_block_timestamp = 1466646588 # block 1, as 0 is off by a lot
timestamp_average_offset = 160.6855883050695 # calculated at 733447
validate_difficulty: bool = True
def __init__(self, path) -> None:
if path == ':memory:':
self.io = BytesIO()
self.path = path
self._size: Optional[int] = None
self.chunk_getter: Optional[Callable] = None
self.executor = ThreadPoolExecutor(1)
self.known_missing_checkpointed_chunks = set()
self.check_chunk_lock = asyncio.Lock()
async def open(self):
if not self.executor:
self.executor = ThreadPoolExecutor(1)
if self.path != ':memory:':
if not os.path.exists(self.path):
self.io = open(self.path, 'w+b')
else:
self.io = open(self.path, 'r+b')
bytes_size = self.io.seek(0, os.SEEK_END)
self._size = bytes_size // self.header_size
max_checkpointed_height = max(self.checkpoints.keys() or [-1]) + 1000
if bytes_size % self.header_size:
log.warning("Reader file size doesnt match header size. Repairing, might take a while.")
await self.repair()
else:
# try repairing any incomplete write on tip from previous runs (outside of checkpoints, that are ok)
await self.repair(start_height=max_checkpointed_height)
await self.ensure_checkpointed_size()
await self.get_all_missing_headers()
async def close(self):
if self.executor:
self.executor.shutdown()
self.executor = None
self.io.close()
@staticmethod
def serialize(header):
return b''.join([
struct.pack('<I', header['version']),
unhexlify(header['prev_block_hash'])[::-1],
unhexlify(header['merkle_root'])[::-1],
unhexlify(header['claim_trie_root'])[::-1],
struct.pack('<III', header['timestamp'], header['bits'], header['nonce'])
])
@staticmethod
def deserialize(height, header):
version, = struct.unpack('<I', header[:4])
timestamp, bits, nonce = struct.unpack('<III', header[100:112])
return {
'version': version,
'prev_block_hash': hexlify(header[4:36][::-1]),
'merkle_root': hexlify(header[36:68][::-1]),
'claim_trie_root': hexlify(header[68:100][::-1]),
'timestamp': timestamp,
'bits': bits,
'nonce': nonce,
'block_height': height,
}
def get_next_chunk_target(self, chunk: int) -> ArithUint256:
return ArithUint256(self.max_target)
def get_next_block_target(self, max_target: ArithUint256, previous: Optional[dict],
current: Optional[dict]) -> ArithUint256:
# https://github.com/lbryio/lbrycrd/blob/master/src/lbry.cpp
if previous is None and current is None:
return max_target
if previous is None:
previous = current
actual_timespan = current['timestamp'] - previous['timestamp']
modulated_timespan = self.target_timespan + int((actual_timespan - self.target_timespan) / 8)
minimum_timespan = self.target_timespan - int(self.target_timespan / 8) # 150 - 18 = 132
maximum_timespan = self.target_timespan + int(self.target_timespan / 2) # 150 + 75 = 225
clamped_timespan = max(minimum_timespan, min(modulated_timespan, maximum_timespan))
target = ArithUint256.from_compact(current['bits'])
new_target = min(max_target, (target * clamped_timespan) / self.target_timespan)
return new_target
def __len__(self) -> int:
return self._size
def __bool__(self):
return True
async def get(self, height) -> dict:
if isinstance(height, slice):
raise NotImplementedError("Slicing of header chain has not been implemented yet.")
try:
return self.deserialize(height, await self.get_raw_header(height))
except struct.error:
raise IndexError(f"failed to get {height}, at {len(self)}")
def estimated_timestamp(self, height):
if height <= 0:
return
return int(self.first_block_timestamp + (height * self.timestamp_average_offset))
def estimated_date(self, height):
return date.fromtimestamp(self.estimated_timestamp(height))
async def get_raw_header(self, height) -> bytes:
if self.chunk_getter:
await self.ensure_chunk_at(height)
if not 0 <= height <= self.height:
raise IndexError(f"{height} is out of bounds, current height: {self.height}")
return await asyncio.get_running_loop().run_in_executor(self.executor, self._read, height)
def _read(self, height, count=1):
self.io.seek(height * self.header_size, os.SEEK_SET)
return self.io.read(self.header_size * count)
def chunk_hash(self, start, count):
self.io.seek(start * self.header_size, os.SEEK_SET)
return self.hash_header(self.io.read(count * self.header_size)).decode()
async def ensure_checkpointed_size(self):
max_checkpointed_height = max(self.checkpoints.keys() or [-1])
if self.height < max_checkpointed_height:
self._write(max_checkpointed_height, bytes([0] * self.header_size * 1000))
async def ensure_chunk_at(self, height):
async with self.check_chunk_lock:
if await self.has_header(height):
log.debug("has header %s", height)
return
return await self.fetch_chunk(height)
async def fetch_chunk(self, height):
log.info("on-demand fetching height %s", height)
start = (height // 1000) * 1000
headers = await self.chunk_getter(start) # pylint: disable=not-callable
chunk = (
zlib.decompress(base64.b64decode(headers['base64']), wbits=-15, bufsize=600_000)
)
chunk_hash = self.hash_header(chunk).decode()
if self.checkpoints.get(start) == chunk_hash:
await asyncio.get_running_loop().run_in_executor(self.executor, self._write, start, chunk)
if start in self.known_missing_checkpointed_chunks:
self.known_missing_checkpointed_chunks.remove(start)
return
elif start not in self.checkpoints:
return # todo: fixme
raise Exception(
f"Checkpoint mismatch at height {start}. Expected {self.checkpoints[start]}, but got {chunk_hash} instead."
)
async def has_header(self, height):
normalized_height = (height // 1000) * 1000
if normalized_height in self.checkpoints:
return normalized_height not in self.known_missing_checkpointed_chunks
def _has_header(height):
empty = '56944c5d3f98413ef45cf54545538103cc9f298e0575820ad3591376e2e0f65d'
all_zeroes = '789d737d4f448e554b318c94063bbfa63e9ccda6e208f5648ca76ee68896557b'
return self.chunk_hash(height, 1) not in (empty, all_zeroes)
return await asyncio.get_running_loop().run_in_executor(self.executor, _has_header, height)
async def get_all_missing_headers(self):
# Heavy operation done in one optimized shot
def _io_checkall():
for chunk_height, expected_hash in reversed(list(self.checkpoints.items())):
if chunk_height in self.known_missing_checkpointed_chunks:
continue
if self.chunk_hash(chunk_height, 1000) != expected_hash:
self.known_missing_checkpointed_chunks.add(chunk_height)
return self.known_missing_checkpointed_chunks
return await asyncio.get_running_loop().run_in_executor(self.executor, _io_checkall)
@property
def height(self) -> int:
return len(self)-1
@property
def bytes_size(self):
return len(self) * self.header_size
async def hash(self, height=None) -> bytes:
return self.hash_header(
await self.get_raw_header(height if height is not None else self.height)
)
@staticmethod
def hash_header(header: bytes) -> bytes:
if header is None:
return b'0' * 64
return hexlify(double_sha256(header)[::-1])
async def connect(self, start: int, headers: bytes) -> int:
added = 0
bail = False
for height, chunk in self._iterate_chunks(start, headers):
try:
# validate_chunk() is CPU bound and reads previous chunks from file system
await self.validate_chunk(height, chunk)
except InvalidHeader as e:
bail = True
chunk = chunk[:(height-e.height)*self.header_size]
if chunk:
added += await asyncio.get_running_loop().run_in_executor(self.executor, self._write, height, chunk)
if bail:
break
return added
def _write(self, height, verified_chunk):
self.io.seek(height * self.header_size, os.SEEK_SET)
written = self.io.write(verified_chunk) // self.header_size
# self.io.truncate()
# .seek()/.write()/.truncate() might also .flush() when needed
# the goal here is mainly to ensure we're definitely flush()'ing
self.io.flush()
self._size = max(self._size or 0, self.io.tell() // self.header_size)
return written
async def validate_chunk(self, height, chunk):
previous_hash, previous_header, previous_previous_header = None, None, None
if height > 0:
raw = await self.get_raw_header(height-1)
previous_header = self.deserialize(height-1, raw)
previous_hash = self.hash_header(raw)
if height > 1:
previous_previous_header = await self.get(height-2)
chunk_target = self.get_next_chunk_target(height // 2016 - 1)
for current_hash, current_header in self._iterate_headers(height, chunk):
block_target = self.get_next_block_target(chunk_target, previous_previous_header, previous_header)
self.validate_header(height, current_hash, current_header, previous_hash, block_target)
previous_previous_header = previous_header
previous_header = current_header
previous_hash = current_hash
def validate_header(self, height: int, current_hash: bytes,
header: dict, previous_hash: bytes, target: ArithUint256):
if previous_hash is None:
if self.genesis_hash is not None and self.genesis_hash != current_hash:
raise InvalidHeader(
height, f"genesis header doesn't match: {current_hash.decode()} "
f"vs expected {self.genesis_hash.decode()}")
return
if header['prev_block_hash'] != previous_hash:
raise InvalidHeader(
height, "previous hash mismatch: {} vs expected {}".format(
header['prev_block_hash'].decode(), previous_hash.decode())
)
if self.validate_difficulty:
if header['bits'] != target.compact:
raise InvalidHeader(
height, "bits mismatch: {} vs expected {}".format(
header['bits'], target.compact)
)
proof_of_work = self.get_proof_of_work(current_hash)
if proof_of_work > target:
raise InvalidHeader(
height, f"insufficient proof of work: {proof_of_work.value} vs target {target.value}"
)
async def repair(self, start_height=0):
previous_header_hash = fail = None
batch_size = 36
for height in range(start_height, self.height, batch_size):
headers = await asyncio.get_running_loop().run_in_executor(
self.executor, self._read, height, batch_size
)
if len(headers) % self.header_size != 0:
headers = headers[:(len(headers) // self.header_size) * self.header_size]
for header_hash, header in self._iterate_headers(height, headers):
height = header['block_height']
if previous_header_hash:
if header['prev_block_hash'] != previous_header_hash:
fail = True
elif height == 0:
if header_hash != self.genesis_hash:
fail = True
else:
# for sanity and clarity, since it is the only way we can end up here
assert start_height > 0 and height == start_height
if fail:
log.warning("Header file corrupted at height %s, truncating it.", height - 1)
def __truncate(at_height):
self.io.seek(max(0, (at_height - 1)) * self.header_size, os.SEEK_SET)
self.io.truncate()
self.io.flush()
self._size = self.io.seek(0, os.SEEK_END) // self.header_size
return await asyncio.get_running_loop().run_in_executor(self.executor, __truncate, height)
previous_header_hash = header_hash
@classmethod
def get_proof_of_work(cls, header_hash: bytes):
return ArithUint256(int(b'0x' + cls.header_hash_to_pow_hash(header_hash), 16))
def _iterate_chunks(self, height: int, headers: bytes) -> Iterator[Tuple[int, bytes]]:
assert len(headers) % self.header_size == 0, f"{len(headers)} {len(headers)%self.header_size}"
start = 0
end = (self.chunk_size - height % self.chunk_size) * self.header_size
while start < end:
yield height + (start // self.header_size), headers[start:end]
start = end
end = min(len(headers), end + self.chunk_size * self.header_size)
def _iterate_headers(self, height: int, headers: bytes) -> Iterator[Tuple[bytes, dict]]:
assert len(headers) % self.header_size == 0, len(headers)
for idx in range(len(headers) // self.header_size):
start, end = idx * self.header_size, (idx + 1) * self.header_size
header = headers[start:end]
yield self.hash_header(header), self.deserialize(height+idx, header)
@staticmethod
def header_hash_to_pow_hash(header_hash: bytes):
header_hash_bytes = unhexlify(header_hash)[::-1]
h = sha512(header_hash_bytes)
pow_hash = double_sha256(
ripemd160(h[:len(h) // 2]) +
ripemd160(h[len(h) // 2:])
)
return hexlify(pow_hash[::-1])
class UnvalidatedHeaders(Headers):
validate_difficulty = False
max_target = 0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
genesis_hash = b'6e3fcf1299d4ec5d79c3a4c91d624a4acf9e2e173d95a1a0504f677669687556'
checkpoints = {}
+346
View File
@@ -0,0 +1,346 @@
import os
import struct
import shutil
import asyncio
import logging
import zipfile
import tempfile
import urllib.request
from typing import Optional
from binascii import hexlify
import aiohttp
import zmq
import zmq.asyncio
from lbry.conf import Config
from lbry.event import EventController
from lbry.error import LbrycrdEventSubscriptionError, LbrycrdUnauthorizedError
from .database import BlockchainDB
from .ledger import Ledger, RegTestLedger
log = logging.getLogger(__name__)
DOWNLOAD_URL = (
'https://github.com/lbryio/lbrycrd/releases/download/v0.17.4.6/lbrycrd-linux-1746.zip'
)
class Process(asyncio.SubprocessProtocol):
IGNORE_OUTPUT = [
b'keypool keep',
b'keypool reserve',
b'keypool return',
]
def __init__(self):
self.ready = asyncio.Event()
self.stopped = asyncio.Event()
def pipe_data_received(self, fd, data):
if not any(ignore in data for ignore in self.IGNORE_OUTPUT):
if b'Error:' in data:
log.error(data.decode())
else:
for line in data.decode().splitlines():
log.debug(line.rstrip())
if b'Error:' in data:
self.ready.set()
raise SystemError(data.decode())
if b'Done loading' in data:
self.ready.set()
def process_exited(self):
self.stopped.set()
self.ready.set()
ZMQ_BLOCK_EVENT = 'pubhashblock'
class Lbrycrd:
def __init__(self, ledger: Ledger):
self.ledger, self.conf = ledger, ledger.conf
self.data_dir = self.actual_data_dir = ledger.conf.lbrycrd_dir
if self.is_regtest:
self.actual_data_dir = os.path.join(self.data_dir, 'regtest')
self.blocks_dir = os.path.join(self.actual_data_dir, 'blocks')
self.bin_dir = os.path.join(os.path.dirname(__file__), 'bin')
self.daemon_bin = os.path.join(self.bin_dir, 'lbrycrdd')
self.cli_bin = os.path.join(self.bin_dir, 'lbrycrd-cli')
self.protocol = None
self.transport = None
self.subscribed = False
self.subscription: Optional[asyncio.Task] = None
self.default_generate_address = None
self._on_block_hash_controller = EventController()
self.on_block_hash = self._on_block_hash_controller.stream
self.on_block_hash.listen(lambda e: log.info('%s %s', hexlify(e['hash']), e['msg']))
self._on_tx_hash_controller = EventController()
self.on_tx_hash = self._on_tx_hash_controller.stream
self.db = BlockchainDB(self.actual_data_dir)
self._session: Optional[aiohttp.ClientSession] = None
self._loop: Optional[asyncio.AbstractEventLoop] = None
@property
def session(self) -> aiohttp.ClientSession:
if self._session is None:
self._session = aiohttp.ClientSession()
return self._session
@classmethod
def temp_regtest(cls):
return cls(RegTestLedger(
Config.with_same_dir(tempfile.mkdtemp()).set(
blockchain="regtest",
lbrycrd_rpc_port=9245 + 2, # avoid conflict with default rpc port
lbrycrd_peer_port=9246 + 2, # avoid conflict with default peer port
lbrycrd_zmq="tcp://127.0.0.1:29002"
)
))
@staticmethod
def get_block_file_name(block_file_number):
return f'blk{block_file_number:05}.dat'
def get_block_file_path(self, block_file_number):
return os.path.join(
self.actual_data_dir, 'blocks',
self.get_block_file_name(block_file_number)
)
@property
def is_regtest(self):
return isinstance(self.ledger, RegTestLedger)
@property
def rpc_url(self):
return (
f'http://{self.conf.lbrycrd_rpc_user}:{self.conf.lbrycrd_rpc_pass}'
f'@{self.conf.lbrycrd_rpc_host}:{self.conf.lbrycrd_rpc_port}/'
)
@property
def exists(self):
return (
os.path.exists(self.cli_bin) and
os.path.exists(self.daemon_bin)
)
async def download(self):
downloaded_file = os.path.join(
self.bin_dir, DOWNLOAD_URL[DOWNLOAD_URL.rfind('/')+1:]
)
if not os.path.exists(self.bin_dir):
os.mkdir(self.bin_dir)
if not os.path.exists(downloaded_file):
log.info('Downloading: %s', DOWNLOAD_URL)
async with aiohttp.ClientSession() as session:
async with session.get(DOWNLOAD_URL) as response:
with open(downloaded_file, 'wb') as out_file:
while True:
chunk = await response.content.read(4096)
if not chunk:
break
out_file.write(chunk)
with urllib.request.urlopen(DOWNLOAD_URL) as response:
with open(downloaded_file, 'wb') as out_file:
shutil.copyfileobj(response, out_file)
log.info('Extracting: %s', downloaded_file)
with zipfile.ZipFile(downloaded_file) as dotzip:
dotzip.extractall(self.bin_dir)
# zipfile bug https://bugs.python.org/issue15795
os.chmod(self.cli_bin, 0o755)
os.chmod(self.daemon_bin, 0o755)
return self.exists
async def ensure(self):
return self.exists or await self.download()
def get_start_command(self, *args):
if self.is_regtest:
args += ('-regtest',)
if self.conf.lbrycrd_zmq:
args += (
f'-zmqpubhashblock={self.conf.lbrycrd_zmq}',
f'-zmqpubhashtx={self.conf.lbrycrd_zmq}',
)
return (
self.daemon_bin,
f'-datadir={self.data_dir}',
f'-port={self.conf.lbrycrd_peer_port}',
f'-rpcport={self.conf.lbrycrd_rpc_port}',
f'-rpcuser={self.conf.lbrycrd_rpc_user}',
f'-rpcpassword={self.conf.lbrycrd_rpc_pass}',
'-server', '-printtoconsole',
*args
)
async def open(self):
await self.db.open()
async def close(self):
await self.db.close()
await self.close_session()
async def close_session(self):
if self._session is not None:
await self._session.close()
self._session = None
async def start(self, *args):
loop = asyncio.get_running_loop()
command = self.get_start_command(*args)
log.info(' '.join(command))
self.transport, self.protocol = await loop.subprocess_exec(Process, *command)
await self.protocol.ready.wait()
assert not self.protocol.stopped.is_set()
await self.open()
async def stop(self, cleanup=True):
try:
await self.close()
self.transport.terminate()
await self.protocol.stopped.wait()
assert self.transport.get_returncode() == 0, "lbrycrd daemon exit with error"
self.transport.close()
finally:
if cleanup:
await self.cleanup()
async def cleanup(self):
await asyncio.get_running_loop().run_in_executor(
None, shutil.rmtree, self.data_dir, True
)
async def ensure_subscribable(self):
zmq_notifications = await self.get_zmq_notifications()
subs = {e['type']: e['address'] for e in zmq_notifications}
if ZMQ_BLOCK_EVENT not in subs:
raise LbrycrdEventSubscriptionError(ZMQ_BLOCK_EVENT)
if not self.conf.lbrycrd_zmq:
self.conf.lbrycrd_zmq = subs[ZMQ_BLOCK_EVENT]
async def subscribe(self):
if not self.subscribed:
self.subscribed = True
ctx = zmq.asyncio.Context.instance()
sock = ctx.socket(zmq.SUB) # pylint: disable=no-member
sock.connect(self.conf.lbrycrd_zmq)
sock.subscribe("hashblock")
sock.subscribe("hashtx")
self.subscription = asyncio.create_task(self.subscription_handler(sock))
async def subscription_handler(self, sock):
try:
while self.subscribed:
msg = await sock.recv_multipart()
if msg[0] == b'hashtx':
await self._on_tx_hash_controller.add({
'hash': msg[1],
'msg': struct.unpack('<I', msg[2])[0]
})
elif msg[0] == b'hashblock':
await self._on_block_hash_controller.add({
'hash': msg[1],
'msg': struct.unpack('<I', msg[2])[0]
})
except asyncio.CancelledError:
sock.close()
raise
def unsubscribe(self):
if self.subscribed:
self.subscribed = False
self.subscription.cancel()
self.subscription = None
def sync_run(self, coro):
if self._loop is None:
try:
self._loop = asyncio.get_event_loop()
except RuntimeError:
self._loop = asyncio.new_event_loop()
return self._loop.run_until_complete(coro)
async def rpc(self, method, params=None):
if self._session is not None and self._session.closed:
raise Exception("session is closed! RPC attempted during shutting down.")
message = {
"jsonrpc": "1.0",
"id": "1",
"method": method,
"params": params or []
}
async with self.session.post(self.rpc_url, json=message) as resp:
if resp.status == 401:
raise LbrycrdUnauthorizedError()
try:
result = await resp.json()
except aiohttp.ContentTypeError as e:
raise Exception(await resp.text()) from e
if not result['error']:
return result['result']
else:
result['error'].update(method=method, params=params)
raise Exception(result['error'])
async def get_zmq_notifications(self):
return await self.rpc("getzmqnotifications")
async def generate(self, blocks):
if self.default_generate_address is None:
self.default_generate_address = await self.get_new_address()
return await self.generate_to_address(blocks, self.default_generate_address)
async def get_new_address(self):
return await self.rpc("getnewaddress")
async def generate_to_address(self, blocks, address):
return await self.rpc("generatetoaddress", [blocks, address])
async def send_to_address(self, address, amount):
return await self.rpc("sendtoaddress", [address, amount])
async def get_block(self, block_hash):
return await self.rpc("getblock", [block_hash])
async def get_raw_mempool(self):
return await self.rpc("getrawmempool")
async def get_raw_transaction(self, txid):
return await self.rpc("getrawtransaction", [txid])
async def fund_raw_transaction(self, tx):
return await self.rpc("fundrawtransaction", [tx])
async def sign_raw_transaction_with_wallet(self, tx):
return await self.rpc("signrawtransactionwithwallet", [tx])
async def send_raw_transaction(self, tx):
return await self.rpc("sendrawtransaction", [tx])
async def claim_name(self, name, data, amount):
return await self.rpc("claimname", [name, data, amount])
async def update_claim(self, txid, data, amount):
return await self.rpc("updateclaim", [txid, data, amount])
async def abandon_claim(self, txid, address):
return await self.rpc("abandonclaim", [txid, address])
async def support_claim(self, name, claim_id, amount, value="", istip=False):
return await self.rpc("supportclaim", [name, claim_id, amount, value, istip])
async def abandon_support(self, txid, address):
return await self.rpc("abandonsupport", [txid, address])
+179
View File
@@ -0,0 +1,179 @@
from binascii import unhexlify
from string import hexdigits
from typing import TYPE_CHECKING, Type
from lbry.crypto.hash import hash160, double_sha256
from lbry.crypto.base58 import Base58
from lbry.schema.url import URL
from .header import Headers, UnvalidatedHeaders
from .checkpoints import HASHES
from .dewies import lbc_to_dewies
if TYPE_CHECKING:
from lbry.conf import Config
class Ledger:
name = 'LBRY Credits'
symbol = 'LBC'
network_name = 'mainnet'
headers_class = Headers
secret_prefix = bytes((0x1c,))
pubkey_address_prefix = bytes((0x55,))
script_address_prefix = bytes((0x7a,))
extended_public_key_prefix = unhexlify('0488b21e')
extended_private_key_prefix = unhexlify('0488ade4')
max_target = 0x0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
genesis_hash = '9c89283ba0f3227f6c03b70216b9f665f0118d5e0fa729cedf4fb34d6a34f463'
genesis_bits = 0x1f00ffff
target_timespan = 150
fee_per_byte = 50
fee_per_name_char = 200000
checkpoints = HASHES
def __init__(self, conf: 'Config'):
self.conf = conf
self.coin_selection_strategy = None
@classmethod
def get_id(cls):
return '{}_{}'.format(cls.symbol.lower(), cls.network_name.lower())
@staticmethod
def address_to_hash160(address) -> bytes:
return Base58.decode(address)[1:21]
@classmethod
def pubkey_hash_to_address(cls, h160):
raw_address = cls.pubkey_address_prefix + h160
return Base58.encode(bytearray(raw_address + double_sha256(raw_address)[0:4]))
@classmethod
def public_key_to_address(cls, public_key):
return cls.pubkey_hash_to_address(hash160(public_key))
@classmethod
def script_hash_to_address(cls, h160):
raw_address = cls.script_address_prefix + h160
return Base58.encode(bytearray(raw_address + double_sha256(raw_address)[0:4]))
@staticmethod
def private_key_to_wif(private_key):
return b'\x1c' + private_key + b'\x01'
@classmethod
def is_valid_address(cls, address):
decoded = Base58.decode_check(address)
return decoded[0] == cls.pubkey_address_prefix[0]
@classmethod
def valid_address_or_error(cls, address):
try:
assert cls.is_valid_address(address)
except:
raise Exception(f"'{address}' is not a valid address")
@staticmethod
def valid_claim_id(claim_id: str):
if not len(claim_id) == 40:
raise Exception(f"Incorrect claimid length: {len(claim_id)}")
if set(claim_id).difference(hexdigits):
raise Exception("Claim id is not hex encoded")
@staticmethod
def valid_channel_name_or_error(name: str):
try:
if not name:
raise Exception("Channel name cannot be blank.")
parsed = URL.parse(name)
if not parsed.has_channel:
raise Exception("Channel names must start with '@' symbol.")
if parsed.channel.name != name:
raise Exception("Channel name has invalid character")
except (TypeError, ValueError):
raise Exception("Invalid channel name.")
@staticmethod
def valid_stream_name_or_error(name: str):
try:
if not name:
raise Exception('Stream name cannot be blank.')
parsed = URL.parse(name)
if parsed.has_channel:
raise Exception(
"Stream names cannot start with '@' symbol. This is reserved for channels claims."
)
if not parsed.has_stream or parsed.stream.name != name:
raise Exception('Stream name has invalid characters.')
except (TypeError, ValueError):
raise Exception("Invalid stream name.")
@staticmethod
def valid_collection_name_or_error(name: str):
try:
if not name:
raise Exception('Collection name cannot be blank.')
parsed = URL.parse(name)
if parsed.has_channel:
raise Exception(
"Collection names cannot start with '@' symbol. This is reserved for channels claims."
)
if not parsed.has_stream or parsed.stream.name != name:
raise Exception('Collection name has invalid characters.')
except (TypeError, ValueError):
raise Exception("Invalid collection name.")
@staticmethod
def get_dewies_or_error(argument: str, lbc: str, positive_value=False):
try:
dewies = lbc_to_dewies(lbc)
if positive_value and dewies <= 0:
raise ValueError(f"'{argument}' value must be greater than 0.0")
return dewies
except ValueError as e:
raise ValueError(f"Invalid value for '{argument}': {e.args[0]}")
def get_fee_address(self, kwargs: dict, claim_address: str) -> str:
if 'fee_address' in kwargs:
self.valid_address_or_error(kwargs['fee_address'])
return kwargs['fee_address']
if 'fee_currency' in kwargs or 'fee_amount' in kwargs:
return claim_address
class TestNetLedger(Ledger):
network_name = 'testnet'
pubkey_address_prefix = bytes((111,))
script_address_prefix = bytes((196,))
extended_public_key_prefix = unhexlify('043587cf')
extended_private_key_prefix = unhexlify('04358394')
checkpoints = {}
class RegTestLedger(Ledger):
network_name = 'regtest'
headers_class = UnvalidatedHeaders
pubkey_address_prefix = bytes((111,))
script_address_prefix = bytes((196,))
extended_public_key_prefix = unhexlify('043587cf')
extended_private_key_prefix = unhexlify('04358394')
max_target = 0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
genesis_hash = '6e3fcf1299d4ec5d79c3a4c91d624a4acf9e2e173d95a1a0504f677669687556'
genesis_bits = 0x207fffff
target_timespan = 1
checkpoints = {}
def ledger_class_from_name(name) -> Type[Ledger]:
return {
Ledger.network_name: Ledger,
TestNetLedger.network_name: TestNetLedger,
RegTestLedger.network_name: RegTestLedger
}[name]
+626
View File
@@ -0,0 +1,626 @@
from typing import List
from itertools import chain
from binascii import hexlify
from collections import namedtuple
from .bcd_data_stream import BCDataStream
def subclass_tuple(name, base):
return type(name, (base,), {'__slots__': ()})
# bitcoin opcodes
OP_0 = 0x00
OP_1 = 0x51
OP_16 = 0x60
OP_VERIFY = 0x69
OP_DUP = 0x76
OP_HASH160 = 0xa9
OP_EQUALVERIFY = 0x88
OP_CHECKSIG = 0xac
OP_CHECKMULTISIG = 0xae
OP_EQUAL = 0x87
OP_PUSHDATA1 = 0x4c
OP_PUSHDATA2 = 0x4d
OP_PUSHDATA4 = 0x4e
OP_RETURN = 0x6a
OP_2DROP = 0x6d
OP_DROP = 0x75
# lbry custom opcodes
# checks
OP_PRICECHECK = 0xb0 # checks that the BUY output is >= SELL price
# tx types
OP_CLAIM_NAME = 0xb5
OP_SUPPORT_CLAIM = 0xb6
OP_UPDATE_CLAIM = 0xb7
OP_SELL_CLAIM = 0xb8
OP_BUY_CLAIM = 0xb9
# template matching opcodes (not real opcodes)
# base class for PUSH_DATA related opcodes
# pylint: disable=invalid-name
PUSH_DATA_OP = namedtuple('PUSH_DATA_OP', 'name')
# opcode for variable length strings
# pylint: disable=invalid-name
PUSH_SINGLE = subclass_tuple('PUSH_SINGLE', PUSH_DATA_OP)
# opcode for variable size integers
# pylint: disable=invalid-name
PUSH_INTEGER = subclass_tuple('PUSH_INTEGER', PUSH_DATA_OP)
# opcode for variable number of variable length strings
# pylint: disable=invalid-name
PUSH_MANY = subclass_tuple('PUSH_MANY', PUSH_DATA_OP)
# opcode with embedded subscript parsing
# pylint: disable=invalid-name
PUSH_SUBSCRIPT = namedtuple('PUSH_SUBSCRIPT', 'name template')
def is_push_data_opcode(opcode):
return isinstance(opcode, (PUSH_DATA_OP, PUSH_SUBSCRIPT))
def is_push_data_token(token):
return 1 <= token <= OP_PUSHDATA4
def push_data(data):
size = len(data)
if size < OP_PUSHDATA1:
yield BCDataStream.uint8.pack(size)
elif size <= 0xFF:
yield BCDataStream.uint8.pack(OP_PUSHDATA1)
yield BCDataStream.uint8.pack(size)
elif size <= 0xFFFF:
yield BCDataStream.uint8.pack(OP_PUSHDATA2)
yield BCDataStream.uint16.pack(size)
else:
yield BCDataStream.uint8.pack(OP_PUSHDATA4)
yield BCDataStream.uint32.pack(size)
yield bytes(data)
def read_data(token, stream):
if token < OP_PUSHDATA1:
return stream.read(token)
if token == OP_PUSHDATA1:
return stream.read(stream.read_uint8())
if token == OP_PUSHDATA2:
return stream.read(stream.read_uint16())
return stream.read(stream.read_uint32())
# opcode for OP_1 - OP_16
# pylint: disable=invalid-name
SMALL_INTEGER = namedtuple('SMALL_INTEGER', 'name')
def is_small_integer(token):
return OP_1 <= token <= OP_16
def push_small_integer(num):
assert 1 <= num <= 16
yield BCDataStream.uint8.pack(OP_1 + (num - 1))
def read_small_integer(token):
return (token - OP_1) + 1
class Token(namedtuple('Token', 'value')):
__slots__ = ()
def __repr__(self):
name = None
for var_name, var_value in globals().items():
if var_name.startswith('OP_') and var_value == self.value:
name = var_name
break
return name or self.value
class DataToken(Token):
__slots__ = ()
def __repr__(self):
return f'"{hexlify(self.value)}"'
class SmallIntegerToken(Token):
__slots__ = ()
def __repr__(self):
return f'SmallIntegerToken({self.value})'
def token_producer(source):
token = source.read_uint8()
while token is not None:
if is_push_data_token(token):
yield DataToken(read_data(token, source))
elif is_small_integer(token):
yield SmallIntegerToken(read_small_integer(token))
else:
yield Token(token)
token = source.read_uint8()
def tokenize(source):
return list(token_producer(source))
class ScriptError(Exception):
""" General script handling error. """
class ParseError(ScriptError):
""" Script parsing error. """
class Parser:
def __init__(self, opcodes, tokens):
self.opcodes = opcodes
self.tokens = tokens
self.values = {}
self.token_index = 0
self.opcode_index = 0
def parse(self):
while self.token_index < len(self.tokens) and self.opcode_index < len(self.opcodes):
token = self.tokens[self.token_index]
opcode = self.opcodes[self.opcode_index]
if token.value == 0 and isinstance(opcode, PUSH_SINGLE):
token = DataToken(b'')
if isinstance(token, DataToken):
if isinstance(opcode, (PUSH_SINGLE, PUSH_INTEGER, PUSH_SUBSCRIPT)):
self.push_single(opcode, token.value)
elif isinstance(opcode, PUSH_MANY):
self.consume_many_non_greedy()
else:
raise ParseError(f"DataToken found but opcode was '{opcode}'.")
elif isinstance(token, SmallIntegerToken):
if isinstance(opcode, SMALL_INTEGER):
self.values[opcode.name] = token.value
else:
raise ParseError(f"SmallIntegerToken found but opcode was '{opcode}'.")
elif token.value == opcode:
pass
else:
raise ParseError(f"Token is '{token.value}' and opcode is '{opcode}'.")
self.token_index += 1
self.opcode_index += 1
if self.token_index < len(self.tokens):
raise ParseError("Parse completed without all tokens being consumed.")
if self.opcode_index < len(self.opcodes):
raise ParseError("Parse completed without all opcodes being consumed.")
return self
def consume_many_non_greedy(self):
""" Allows PUSH_MANY to consume data without being greedy
in cases when one or more PUSH_SINGLEs follow a PUSH_MANY. This will
prioritize giving all PUSH_SINGLEs some data and only after that
subsume the rest into PUSH_MANY.
"""
token_values = []
while self.token_index < len(self.tokens):
token = self.tokens[self.token_index]
if not isinstance(token, DataToken):
self.token_index -= 1
break
token_values.append(token.value)
self.token_index += 1
push_opcodes = []
push_many_count = 0
while self.opcode_index < len(self.opcodes):
opcode = self.opcodes[self.opcode_index]
if not is_push_data_opcode(opcode):
self.opcode_index -= 1
break
if isinstance(opcode, PUSH_MANY):
push_many_count += 1
push_opcodes.append(opcode)
self.opcode_index += 1
if push_many_count > 1:
raise ParseError(
"Cannot have more than one consecutive PUSH_MANY, as there is no way to tell which"
" token value should go into which PUSH_MANY."
)
if len(push_opcodes) > len(token_values):
raise ParseError(
"Not enough token values to match all of the PUSH_MANY and PUSH_SINGLE opcodes."
)
many_opcode = push_opcodes.pop(0)
# consume data into PUSH_SINGLE opcodes, working backwards
for opcode in reversed(push_opcodes):
self.push_single(opcode, token_values.pop())
# finally PUSH_MANY gets everything that's left
self.values[many_opcode.name] = token_values
def push_single(self, opcode, value):
if isinstance(opcode, PUSH_SINGLE):
self.values[opcode.name] = value
elif isinstance(opcode, PUSH_INTEGER):
self.values[opcode.name] = int.from_bytes(value, 'little')
elif isinstance(opcode, PUSH_SUBSCRIPT):
self.values[opcode.name] = Script.from_source_with_template(value, opcode.template)
else:
raise ParseError(f"Not a push single or subscript: {opcode}")
class Template:
__slots__ = 'name', 'opcodes'
def __init__(self, name, opcodes):
self.name = name
self.opcodes = opcodes
def parse(self, tokens):
return Parser(self.opcodes, tokens).parse().values if self.opcodes else {}
def generate(self, values):
source = BCDataStream()
for opcode in self.opcodes:
if isinstance(opcode, PUSH_SINGLE):
data = values[opcode.name]
source.write_many(push_data(data))
elif isinstance(opcode, PUSH_INTEGER):
data = values[opcode.name]
source.write_many(push_data(
data.to_bytes((data.bit_length() + 7) // 8, byteorder='little')
))
elif isinstance(opcode, PUSH_SUBSCRIPT):
data = values[opcode.name]
source.write_many(push_data(data.source))
elif isinstance(opcode, PUSH_MANY):
for data in values[opcode.name]:
source.write_many(push_data(data))
elif isinstance(opcode, SMALL_INTEGER):
data = values[opcode.name]
source.write_many(push_small_integer(data))
else:
source.write_uint8(opcode)
return source.get_bytes()
class Script:
__slots__ = 'source', 'offset', '_template', '_values', '_template_hint'
templates: List[Template] = []
NO_SCRIPT = Template('no_script', None) # special case
def __init__(self, source=None, template=None, values=None, template_hint=None, offset=None):
self.source = source
self.offset = offset
self._template = template
self._values = values
self._template_hint = template_hint
if source is None and template and values:
self.generate()
@property
def length(self):
return len(self.source)
@property
def template(self):
if self._template is None:
self.parse(self._template_hint)
return self._template
@property
def values(self):
if self._values is None:
self.parse(self._template_hint)
return self._values
@property
def tokens(self):
return tokenize(BCDataStream(self.source))
@classmethod
def from_source_with_template(cls, source, template):
return cls(source, template_hint=template)
def parse(self, template_hint=None):
tokens = self.tokens
if not tokens and not template_hint:
template_hint = self.NO_SCRIPT
for template in chain((template_hint,), self.templates):
if not template:
continue
try:
self._values = template.parse(tokens)
self._template = template
return
except ParseError:
continue
raise ValueError(f'No matching templates for source: {hexlify(self.source)}')
def generate(self):
self.source = self.template.generate(self._values)
class InputScript(Script):
__slots__ = ()
REDEEM_PUBKEY = Template('pubkey', (
PUSH_SINGLE('signature'),
))
REDEEM_PUBKEY_HASH = Template('pubkey_hash', (
PUSH_SINGLE('signature'), PUSH_SINGLE('pubkey')
))
REDEEM_SCRIPT = Template('script', (
SMALL_INTEGER('signatures_count'), PUSH_MANY('pubkeys'), SMALL_INTEGER('pubkeys_count'),
OP_CHECKMULTISIG
))
REDEEM_SCRIPT_HASH = Template('script_hash', (
OP_0, PUSH_MANY('signatures'), PUSH_SUBSCRIPT('script', REDEEM_SCRIPT)
))
templates = [
REDEEM_PUBKEY,
REDEEM_PUBKEY_HASH,
REDEEM_SCRIPT_HASH,
REDEEM_SCRIPT
]
@classmethod
def redeem_pubkey_hash(cls, signature, pubkey):
return cls(template=cls.REDEEM_PUBKEY_HASH, values={
'signature': signature,
'pubkey': pubkey
})
@classmethod
def redeem_script_hash(cls, signatures, pubkeys):
return cls(template=cls.REDEEM_SCRIPT_HASH, values={
'signatures': signatures,
'script': cls.redeem_script(signatures, pubkeys)
})
@classmethod
def redeem_script(cls, signatures, pubkeys):
return cls(template=cls.REDEEM_SCRIPT, values={
'signatures_count': len(signatures),
'pubkeys': pubkeys,
'pubkeys_count': len(pubkeys)
})
class OutputScript(Script):
__slots__ = ()
# output / payment script templates (aka scriptPubKey)
PAY_PUBKEY_FULL = Template('pay_pubkey_full', (
PUSH_SINGLE('pubkey'), OP_CHECKSIG
))
PAY_PUBKEY_HASH = Template('pay_pubkey_hash', (
OP_DUP, OP_HASH160, PUSH_SINGLE('pubkey_hash'), OP_EQUALVERIFY, OP_CHECKSIG
))
PAY_SCRIPT_HASH = Template('pay_script_hash', (
OP_HASH160, PUSH_SINGLE('script_hash'), OP_EQUAL
))
PAY_SEGWIT = Template('pay_script_hash+segwit', (
OP_0, PUSH_SINGLE('script_hash')
))
RETURN_DATA = Template('return_data', (
OP_RETURN, PUSH_SINGLE('data')
))
CLAIM_NAME_OPCODES = (
OP_CLAIM_NAME, PUSH_SINGLE('claim_name'), PUSH_SINGLE('claim'),
OP_2DROP, OP_DROP
)
CLAIM_NAME_PUBKEY = Template('claim_name+pay_pubkey_hash', (
CLAIM_NAME_OPCODES + PAY_PUBKEY_HASH.opcodes
))
CLAIM_NAME_SCRIPT = Template('claim_name+pay_script_hash', (
CLAIM_NAME_OPCODES + PAY_SCRIPT_HASH.opcodes
))
SUPPORT_CLAIM_OPCODES = (
OP_SUPPORT_CLAIM, PUSH_SINGLE('claim_name'), PUSH_SINGLE('claim_id'),
OP_2DROP, OP_DROP
)
SUPPORT_CLAIM_PUBKEY = Template('support_claim+pay_pubkey_hash', (
SUPPORT_CLAIM_OPCODES + PAY_PUBKEY_HASH.opcodes
))
SUPPORT_CLAIM_SCRIPT = Template('support_claim+pay_script_hash', (
SUPPORT_CLAIM_OPCODES + PAY_SCRIPT_HASH.opcodes
))
SUPPORT_CLAIM_DATA_OPCODES = (
OP_SUPPORT_CLAIM, PUSH_SINGLE('claim_name'), PUSH_SINGLE('claim_id'), PUSH_SINGLE('support'),
OP_2DROP, OP_2DROP
)
SUPPORT_CLAIM_DATA_PUBKEY = Template('support_claim+data+pay_pubkey_hash', (
SUPPORT_CLAIM_DATA_OPCODES + PAY_PUBKEY_HASH.opcodes
))
SUPPORT_CLAIM_DATA_SCRIPT = Template('support_claim+data+pay_script_hash', (
SUPPORT_CLAIM_DATA_OPCODES + PAY_SCRIPT_HASH.opcodes
))
UPDATE_CLAIM_OPCODES = (
OP_UPDATE_CLAIM, PUSH_SINGLE('claim_name'), PUSH_SINGLE('claim_id'), PUSH_SINGLE('claim'),
OP_2DROP, OP_2DROP
)
UPDATE_CLAIM_PUBKEY = Template('update_claim+pay_pubkey_hash', (
UPDATE_CLAIM_OPCODES + PAY_PUBKEY_HASH.opcodes
))
UPDATE_CLAIM_SCRIPT = Template('update_claim+pay_script_hash', (
UPDATE_CLAIM_OPCODES + PAY_SCRIPT_HASH.opcodes
))
SELL_SCRIPT = Template('sell_script', (
OP_VERIFY, OP_DROP, OP_DROP, OP_DROP, PUSH_INTEGER('price'), OP_PRICECHECK
))
SELL_CLAIM = Template('sell_claim+pay_script_hash', (
OP_SELL_CLAIM, PUSH_SINGLE('claim_id'), PUSH_SUBSCRIPT('sell_script', SELL_SCRIPT),
PUSH_SUBSCRIPT('receive_script', InputScript.REDEEM_SCRIPT), OP_2DROP, OP_2DROP
) + PAY_SCRIPT_HASH.opcodes)
BUY_CLAIM = Template('buy_claim+pay_script_hash', (
OP_BUY_CLAIM, PUSH_SINGLE('sell_id'),
PUSH_SINGLE('claim_id'), PUSH_SINGLE('claim_version'),
PUSH_SINGLE('owner_pubkey_hash'), PUSH_SINGLE('negotiation_signature'),
OP_2DROP, OP_2DROP, OP_2DROP,
) + PAY_SCRIPT_HASH.opcodes)
templates = [
PAY_PUBKEY_FULL,
PAY_PUBKEY_HASH,
PAY_SCRIPT_HASH,
PAY_SEGWIT,
RETURN_DATA,
CLAIM_NAME_PUBKEY,
CLAIM_NAME_SCRIPT,
SUPPORT_CLAIM_PUBKEY,
SUPPORT_CLAIM_SCRIPT,
SUPPORT_CLAIM_DATA_PUBKEY,
SUPPORT_CLAIM_DATA_SCRIPT,
UPDATE_CLAIM_PUBKEY,
UPDATE_CLAIM_SCRIPT,
SELL_CLAIM, SELL_SCRIPT,
BUY_CLAIM,
]
@classmethod
def pay_pubkey_hash(cls, pubkey_hash):
return cls(template=cls.PAY_PUBKEY_HASH, values={
'pubkey_hash': pubkey_hash
})
@classmethod
def pay_script_hash(cls, script_hash):
return cls(template=cls.PAY_SCRIPT_HASH, values={
'script_hash': script_hash
})
@classmethod
def return_data(cls, data):
return cls(template=cls.RETURN_DATA, values={
'data': data
})
@property
def is_pay_pubkey(self):
return self.template.name.endswith('pay_pubkey_full')
@classmethod
def pay_claim_name_pubkey_hash(cls, claim_name, claim, pubkey_hash):
return cls(template=cls.CLAIM_NAME_PUBKEY, values={
'claim_name': claim_name,
'claim': claim,
'pubkey_hash': pubkey_hash
})
@classmethod
def pay_update_claim_pubkey_hash(cls, claim_name, claim_id, claim, pubkey_hash):
return cls(template=cls.UPDATE_CLAIM_PUBKEY, values={
'claim_name': claim_name,
'claim_id': claim_id,
'claim': claim,
'pubkey_hash': pubkey_hash
})
@classmethod
def pay_support_pubkey_hash(cls, claim_name: bytes, claim_id: bytes, pubkey_hash: bytes):
return cls(template=cls.SUPPORT_CLAIM_PUBKEY, values={
'claim_name': claim_name,
'claim_id': claim_id,
'pubkey_hash': pubkey_hash
})
@classmethod
def pay_support_data_pubkey_hash(
cls, claim_name: bytes, claim_id: bytes, support, pubkey_hash: bytes):
return cls(template=cls.SUPPORT_CLAIM_DATA_PUBKEY, values={
'claim_name': claim_name,
'claim_id': claim_id,
'support': support,
'pubkey_hash': pubkey_hash
})
@classmethod
def sell_script(cls, price):
return cls(template=cls.SELL_SCRIPT, values={
'price': price,
})
@classmethod
def sell_claim(cls, claim_id, price, signatures, pubkeys):
return cls(template=cls.SELL_CLAIM, values={
'claim_id': claim_id,
'sell_script': OutputScript.sell_script(price),
'receive_script': InputScript.redeem_script(signatures, pubkeys)
})
@classmethod
def buy_claim(cls, sell_id, claim_id, claim_version, owner_pubkey_hash, negotiation_signature):
return cls(template=cls.BUY_CLAIM, values={
'sell_id': sell_id,
'claim_id': claim_id,
'claim_version': claim_version,
'owner_pubkey_hash': owner_pubkey_hash,
'negotiation_signature': negotiation_signature,
})
@property
def is_pay_pubkey_hash(self):
return self.template.name.endswith('pay_pubkey_hash')
@property
def is_pay_script_hash(self):
return self.template.name.endswith('pay_script_hash')
@property
def is_return_data(self):
return self.template.name.endswith('return_data')
@property
def is_claim_name(self):
return self.template.name.startswith('claim_name+')
@property
def is_update_claim(self):
return self.template.name.startswith('update_claim+')
@property
def is_support_claim(self):
return self.template.name.startswith('support_claim+')
@property
def is_support_claim_data(self):
return self.template.name.startswith('support_claim+data+')
@property
def is_sell_claim(self):
return self.template.name.startswith('sell_claim+')
@property
def is_buy_claim(self):
return self.template.name.startswith('buy_claim+')
@property
def is_claim_involved(self):
return any((
self.is_claim_name, self.is_support_claim, self.is_update_claim,
self.is_sell_claim, self.is_buy_claim
))
+1
View File
@@ -0,0 +1 @@
from .synchronizer import BlockchainSync
+336
View File
@@ -0,0 +1,336 @@
import logging
from binascii import hexlify, unhexlify
from typing import Tuple, List
from sqlalchemy import table, text, func, union, between
from sqlalchemy.future import select
from sqlalchemy.schema import CreateTable
from lbry.db.tables import (
Block as BlockTable, BlockFilter, BlockGroupFilter,
TX, TXFilter, MempoolFilter, TXO, TXI, Claim, Tag, Support
)
from lbry.db.tables import (
pg_add_block_constraints_and_indexes,
pg_add_block_filter_constraints_and_indexes,
pg_add_tx_constraints_and_indexes,
pg_add_tx_filter_constraints_and_indexes,
pg_add_txo_constraints_and_indexes,
pg_add_txi_constraints_and_indexes,
)
from lbry.db.query_context import ProgressContext, event_emitter, context
from lbry.db.sync import set_input_addresses, update_spent_outputs
from lbry.blockchain.transaction import Transaction
from lbry.blockchain.block import Block, create_address_filter
from lbry.blockchain.bcd_data_stream import BCDataStream
from .context import get_or_initialize_lbrycrd
from .filter_builder import FilterBuilder
log = logging.getLogger(__name__)
def get_best_block_height_for_file(file_number):
return context().fetchone(
select(func.coalesce(func.max(BlockTable.c.height), -1).label('height'))
.where(BlockTable.c.file_number == file_number)
)['height']
@event_emitter("blockchain.sync.blocks.file", "blocks", "txs", throttle=100)
def sync_block_file(
file_number: int, start_height: int, txs: int, flush_size: int, p: ProgressContext
):
chain = get_or_initialize_lbrycrd(p.ctx)
new_blocks = chain.db.sync_get_blocks_in_file(file_number, start_height)
if not new_blocks:
return -1
file_name = chain.get_block_file_name(file_number)
p.start(len(new_blocks), txs, progress_id=file_number, label=file_name)
block_file_path = chain.get_block_file_path(file_number)
done_blocks = done_txs = 0
last_block_processed, loader = -1, p.ctx.get_bulk_loader()
with open(block_file_path, "rb") as fp:
stream = BCDataStream(fp=fp)
for done_blocks, block_info in enumerate(new_blocks, start=1):
block_height = block_info["height"]
fp.seek(block_info["data_offset"])
block = Block.from_data_stream(stream, block_height, file_number)
loader.add_block(block)
if len(loader.txs) >= flush_size:
done_txs += loader.flush(TX)
p.step(done_blocks, done_txs)
last_block_processed = block_height
if p.ctx.stop_event.is_set():
return last_block_processed
if loader.txs:
done_txs += loader.flush(TX)
p.step(done_blocks, done_txs)
return last_block_processed
@event_emitter("blockchain.sync.blocks.indexes", "steps")
def blocks_constraints_and_indexes(p: ProgressContext):
p.start(1 + len(pg_add_block_constraints_and_indexes))
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE block;"))
p.step()
for constraint in pg_add_block_constraints_and_indexes:
if p.ctx.is_postgres:
p.ctx.execute(text(constraint))
p.step()
@event_emitter("blockchain.sync.blocks.vacuum", "steps")
def blocks_vacuum(p: ProgressContext):
p.start(1)
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM block;"))
p.step()
@event_emitter("blockchain.sync.spends.main", "steps")
def sync_spends(initial_sync: bool, p: ProgressContext):
if initial_sync:
p.start(
7 +
len(pg_add_tx_constraints_and_indexes) +
len(pg_add_txi_constraints_and_indexes) +
len(pg_add_txo_constraints_and_indexes)
)
# 1. tx table stuff
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE tx;"))
p.step()
for constraint in pg_add_tx_constraints_and_indexes:
if p.ctx.is_postgres:
p.ctx.execute(text(constraint))
p.step()
# A. Update TXIs to have the address of TXO they are spending.
# 2. txi table reshuffling
p.ctx.execute(text("ALTER TABLE txi RENAME TO old_txi;"))
p.ctx.execute(CreateTable(TXI, include_foreign_key_constraints=[]))
if p.ctx.is_postgres:
p.ctx.execute(text("ALTER TABLE txi DROP CONSTRAINT txi_pkey;"))
p.step()
# 3. insert
old_txi = table("old_txi", *(c.copy() for c in TXI.columns)) # pylint: disable=not-an-iterable
columns = [c for c in old_txi.columns if c.name != "address"] + [TXO.c.address]
join_txi_on_txo = old_txi.join(TXO, old_txi.c.txo_hash == TXO.c.txo_hash)
select_txis = select(*columns).select_from(join_txi_on_txo)
insert_txis = TXI.insert().from_select(columns, select_txis)
p.ctx.execute(insert_txis)
p.step()
# 4. drop old txi and vacuum
p.ctx.execute(text("DROP TABLE old_txi;"))
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE txi;"))
p.step()
for constraint in pg_add_txi_constraints_and_indexes:
if p.ctx.is_postgres:
p.ctx.execute(text(constraint))
p.step()
# B. Update TXOs to have the height at which they were spent (if they were).
# 5. txo table reshuffling
p.ctx.execute(text("ALTER TABLE txo RENAME TO old_txo;"))
p.ctx.execute(CreateTable(TXO, include_foreign_key_constraints=[]))
if p.ctx.is_postgres:
p.ctx.execute(text("ALTER TABLE txo DROP CONSTRAINT txo_pkey;"))
p.step()
# 6. insert
old_txo = table("old_txo", *(c.copy() for c in TXO.columns)) # pylint: disable=not-an-iterable
columns = [c for c in old_txo.columns if c.name != "spent_height"]
insert_columns = columns + [TXO.c.spent_height]
select_columns = columns + [func.coalesce(TXI.c.height, 0).label("spent_height")]
join_txo_on_txi = old_txo.join(TXI, old_txo.c.txo_hash == TXI.c.txo_hash, isouter=True)
select_txos = select(*select_columns).select_from(join_txo_on_txi)
insert_txos = TXO.insert().from_select(insert_columns, select_txos)
p.ctx.execute(insert_txos)
p.step()
# 7. drop old txo
p.ctx.execute(text("DROP TABLE old_txo;"))
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE txo;"))
p.step()
for constraint in pg_add_txo_constraints_and_indexes:
if p.ctx.is_postgres:
p.ctx.execute(text(constraint))
p.step()
else:
p.start(5)
# 1. Update spent TXOs setting spent_height
update_spent_outputs(p.ctx)
p.step()
# 2. Update TXIs to have the address of TXO they are spending.
set_input_addresses(p.ctx)
p.step()
# 3. Update tx visibility map, which speeds up index-only scans.
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM tx;"))
p.step()
# 4. Update txi visibility map, which speeds up index-only scans.
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM txi;"))
p.step()
# 4. Update txo visibility map, which speeds up index-only scans.
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM txo;"))
p.step()
@event_emitter("blockchain.sync.mempool.clear", "txs")
def clear_mempool(p: ProgressContext):
delete_all_the_things(-1, p)
@event_emitter("blockchain.sync.mempool.main", "txs")
def sync_mempool(p: ProgressContext) -> List[str]:
chain = get_or_initialize_lbrycrd(p.ctx)
mempool = chain.sync_run(chain.get_raw_mempool())
current = [hexlify(r['tx_hash'][::-1]).decode() for r in p.ctx.fetchall(
select(TX.c.tx_hash).where(TX.c.height < 0)
)]
loader = p.ctx.get_bulk_loader()
added = []
for txid in mempool:
if txid not in current:
raw_tx = chain.sync_run(chain.get_raw_transaction(txid))
loader.add_transaction(
None, Transaction(unhexlify(raw_tx), height=-1)
)
added.append(txid)
if p.ctx.stop_event.is_set():
return
loader.flush(TX)
return added
@event_emitter("blockchain.sync.filters.generate", "blocks", throttle=100)
def sync_filters(start, end, p: ProgressContext):
fp = FilterBuilder(start, end)
p.start((end-start)+1, progress_id=start, label=f"generate filters {start}-{end}")
with p.ctx.connect_streaming() as c:
loader = p.ctx.get_bulk_loader()
tx_hash, height, addresses, last_added = None, None, set(), None
address_to_hash = p.ctx.ledger.address_to_hash160
for row in c.execute(get_block_tx_addresses_sql(*fp.query_heights)):
if tx_hash != row.tx_hash:
if tx_hash is not None:
last_added = tx_hash
fp.add(tx_hash, height, addresses)
tx_hash, height, addresses = row.tx_hash, row.height, set()
addresses.add(address_to_hash(row.address))
if all([last_added, tx_hash]) and last_added != tx_hash: # pickup last tx
fp.add(tx_hash, height, addresses)
for tx_hash, height, addresses in fp.tx_filters:
loader.add_transaction_filter(
tx_hash, height, create_address_filter(list(addresses))
)
for height, addresses in fp.block_filters.items():
loader.add_block_filter(
height, create_address_filter(list(addresses))
)
for group_filter in fp.group_filters:
for height, addresses in group_filter.groups.items():
loader.add_group_filter(
height, group_filter.factor, create_address_filter(list(addresses))
)
p.add(loader.flush(BlockFilter))
@event_emitter("blockchain.sync.filters.indexes", "steps")
def filters_constraints_and_indexes(p: ProgressContext):
constraints = (
pg_add_tx_filter_constraints_and_indexes +
pg_add_block_filter_constraints_and_indexes
)
p.start(2 + len(constraints))
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE block_filter;"))
p.step()
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE tx_filter;"))
p.step()
for constraint in constraints:
if p.ctx.is_postgres:
p.ctx.execute(text(constraint))
p.step()
@event_emitter("blockchain.sync.filters.vacuum", "steps")
def filters_vacuum(p: ProgressContext):
p.start(2)
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM block_filter;"))
p.step()
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM tx_filter;"))
p.step()
def get_block_range_without_filters() -> Tuple[int, int]:
sql = (
select(
func.coalesce(func.min(BlockTable.c.height), -1).label('start_height'),
func.coalesce(func.max(BlockTable.c.height), -1).label('end_height'),
)
.select_from(
BlockTable.join(BlockFilter, BlockTable.c.height == BlockFilter.c.height, isouter=True)
)
.where(BlockFilter.c.height.is_(None))
)
result = context().fetchone(sql)
return result['start_height'], result['end_height']
def get_block_tx_addresses_sql(start_height, end_height):
return union(
select(TXO.c.tx_hash, TXO.c.height, TXO.c.address).where(
(TXO.c.address.isnot(None)) & between(TXO.c.height, start_height, end_height)
),
select(TXI.c.tx_hash, TXI.c.height, TXI.c.address).where(
(TXI.c.address.isnot(None)) & between(TXI.c.height, start_height, end_height)
),
).order_by('height', 'tx_hash')
@event_emitter("blockchain.sync.rewind.main", "steps")
def rewind(height: int, p: ProgressContext):
delete_all_the_things(height, p)
def delete_all_the_things(height: int, p: ProgressContext):
def constrain(col):
if height == -1:
return col == -1
return col >= height
deletes = [
BlockTable.delete().where(constrain(BlockTable.c.height)),
TXI.delete().where(constrain(TXI.c.height)),
TXO.delete().where(constrain(TXO.c.height)),
TX.delete().where(constrain(TX.c.height)),
Tag.delete().where(
Tag.c.claim_hash.in_(
select(Claim.c.claim_hash).where(constrain(Claim.c.height))
)
),
Claim.delete().where(constrain(Claim.c.height)),
Support.delete().where(constrain(Support.c.height)),
MempoolFilter.delete(),
]
if height > 0:
deletes.extend([
BlockFilter.delete().where(BlockFilter.c.height >= height),
# TODO: group and tx filters need where() clauses (below actually breaks things)
BlockGroupFilter.delete(),
TXFilter.delete(),
])
for delete in p.iter(deletes):
p.ctx.execute(delete)
+338
View File
@@ -0,0 +1,338 @@
import logging
from typing import Tuple
from sqlalchemy import case, func, desc, text
from sqlalchemy.future import select
from lbry.db.queries.txio import (
minimum_txo_columns, row_to_txo,
where_unspent_txos, where_claims_with_changed_supports,
count_unspent_txos, where_channels_with_changed_content,
where_abandoned_claims, count_channels_with_changed_content,
where_claims_with_changed_reposts,
)
from lbry.db.query_context import ProgressContext, event_emitter
from lbry.db.tables import (
TX, TXO, Claim, Support, CensoredClaim,
pg_add_claim_and_tag_constraints_and_indexes
)
from lbry.db.utils import least
from lbry.db.constants import TXO_TYPES, CLAIM_TYPE_CODES
from lbry.schema.result import Censor
from lbry.blockchain.transaction import Output
from .context import get_or_initialize_lbrycrd
log = logging.getLogger(__name__)
def channel_content_count_calc(signable):
return (
select(func.count(signable.c.claim_hash))
.where((signable.c.channel_hash == Claim.c.claim_hash) & signable.c.is_signature_valid)
.scalar_subquery()
)
support = TXO.alias('support')
def staked_support_subquery(claim_hash_column, aggregate):
"""Return a query that selects unspent supports"""
content = Claim.alias("content")
return (
select(
aggregate
).select_from(
support
.join(content, support.c.claim_hash == content.c.claim_hash)
).where(
((content.c.claim_hash == claim_hash_column) | (content.c.channel_hash == claim_hash_column)) &
(support.c.txo_type == TXO_TYPES["support"]) &
(support.c.spent_height == 0)
)
.scalar_subquery()
)
def staked_support_amount_calc(claim_hash):
"""Return a query that sums unspent supports for a claim"""
return staked_support_subquery(claim_hash, func.coalesce(func.sum(support.c.amount), 0))
def staked_support_count_calc(claim_hash):
"""Return a query that counts unspent supports for a claim"""
return staked_support_subquery(claim_hash, func.coalesce(func.count('*'), 0))
def claims_in_channel_amount_calc(claim_hash):
"""Return a query that sums the amount of all the claims in a channel"""
content = Claim.alias("content")
return (
select(
func.coalesce(func.sum(content.c.amount), 0)
).select_from(
content
).where(
content.c.channel_hash == claim_hash
)
.scalar_subquery()
)
def reposted_claim_count_calc(other):
repost = TXO.alias('repost')
return (
select(func.coalesce(func.count(repost.c.reposted_claim_hash), 0))
.where(
(repost.c.reposted_claim_hash == other.c.claim_hash) &
(repost.c.spent_height == 0)
).scalar_subquery()
)
def make_label(action, blocks):
if blocks[0] == blocks[-1]:
return f"{action} {blocks[0]:>6}"
else:
return f"{action} {blocks[0]:>6}-{blocks[-1]:>6}"
def select_claims_for_saving(
blocks: Tuple[int, int],
missing_in_claims_table=False,
missing_or_stale_in_claims_table=False,
):
channel_txo = TXO.alias('channel_txo')
return select(
*minimum_txo_columns, TXO.c.claim_hash,
claims_in_channel_amount_calc(TXO.c.claim_hash).label('claims_in_channel_amount'),
staked_support_amount_calc(TXO.c.claim_hash).label('staked_support_amount'),
staked_support_count_calc(TXO.c.claim_hash).label('staked_support_count'),
reposted_claim_count_calc(TXO).label('reposted_count'),
TXO.c.signature, TXO.c.signature_digest,
case([(
TXO.c.channel_hash.isnot(None),
select(channel_txo.c.public_key).select_from(channel_txo).where(
(channel_txo.c.txo_type == TXO_TYPES['channel']) &
(channel_txo.c.claim_hash == TXO.c.channel_hash) &
(channel_txo.c.height <= TXO.c.height)
).order_by(desc(channel_txo.c.height)).limit(1).scalar_subquery()
)]).label('channel_public_key')
).where(
where_unspent_txos(
CLAIM_TYPE_CODES, blocks,
missing_in_claims_table=missing_in_claims_table,
missing_or_stale_in_claims_table=missing_or_stale_in_claims_table,
)
).select_from(TXO.join(TX))
def row_to_claim_for_saving(row) -> Tuple[Output, dict]:
return row_to_txo(row), {
'claims_in_channel_amount': int(row.claims_in_channel_amount),
'staked_support_amount': int(row.staked_support_amount),
'staked_support_count': int(row.staked_support_count),
'reposted_count': int(row.reposted_count),
'signature': row.signature,
'signature_digest': row.signature_digest,
'channel_public_key': row.channel_public_key
}
@event_emitter("blockchain.sync.claims.insert", "claims")
def claims_insert(
blocks: Tuple[int, int],
missing_in_claims_table: bool,
flush_size: int,
p: ProgressContext
):
chain = get_or_initialize_lbrycrd(p.ctx)
p.start(
count_unspent_txos(
CLAIM_TYPE_CODES, blocks,
missing_in_claims_table=missing_in_claims_table,
), progress_id=blocks[0], label=make_label("add claims", blocks)
)
with p.ctx.connect_streaming() as c:
loader = p.ctx.get_bulk_loader()
cursor = c.execute(select_claims_for_saving(
blocks, missing_in_claims_table=missing_in_claims_table
).order_by(TXO.c.claim_hash))
for rows in cursor.partitions(900):
claim_metadata = chain.db.sync_get_claim_metadata(
claim_hashes=[row['claim_hash'] for row in rows]
)
i = 0
for row in rows:
metadata = claim_metadata[i] if i < len(claim_metadata) else {}
if metadata and metadata['claim_hash'] == row.claim_hash:
i += 1
txo, extra = row_to_claim_for_saving(row)
extra.update({
'short_url': metadata.get('short_url'),
'creation_height': metadata.get('creation_height'),
'activation_height': metadata.get('activation_height'),
'expiration_height': metadata.get('expiration_height'),
'takeover_height': metadata.get('takeover_height'),
})
loader.add_claim(txo, **extra)
if len(loader.claims) >= flush_size:
p.add(loader.flush(Claim))
p.add(loader.flush(Claim))
@event_emitter("blockchain.sync.claims.indexes", "steps")
def claims_constraints_and_indexes(p: ProgressContext):
p.start(2 + len(pg_add_claim_and_tag_constraints_and_indexes))
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE claim;"))
p.step()
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE tag;"))
p.step()
for constraint in pg_add_claim_and_tag_constraints_and_indexes:
if p.ctx.is_postgres:
p.ctx.execute(text(constraint))
p.step()
@event_emitter("blockchain.sync.claims.vacuum", "steps")
def claims_vacuum(p: ProgressContext):
p.start(2)
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM claim;"))
p.step()
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM tag;"))
p.step()
@event_emitter("blockchain.sync.claims.update", "claims")
def claims_update(blocks: Tuple[int, int], p: ProgressContext):
p.start(
count_unspent_txos(CLAIM_TYPE_CODES, blocks, missing_or_stale_in_claims_table=True),
progress_id=blocks[0], label=make_label("mod claims", blocks)
)
with p.ctx.connect_streaming() as c:
loader = p.ctx.get_bulk_loader()
cursor = c.execute(select_claims_for_saving(
blocks, missing_or_stale_in_claims_table=True
))
for row in cursor:
txo, extra = row_to_claim_for_saving(row)
loader.update_claim(txo, **extra)
if len(loader.update_claims) >= 25:
p.add(loader.flush(Claim))
p.add(loader.flush(Claim))
@event_emitter("blockchain.sync.claims.delete", "claims")
def claims_delete(claims, p: ProgressContext):
p.start(claims, label="del claims")
deleted = p.ctx.execute(Claim.delete().where(where_abandoned_claims()))
p.step(deleted.rowcount)
@event_emitter("blockchain.sync.claims.takeovers", "claims")
def update_takeovers(blocks: Tuple[int, int], takeovers, p: ProgressContext):
p.start(takeovers, label=make_label("mod winner", blocks))
chain = get_or_initialize_lbrycrd(p.ctx)
with p.ctx.engine.begin() as c:
for takeover in chain.db.sync_get_takeovers(start_height=blocks[0], end_height=blocks[-1]):
update_claims = (
Claim.update()
.where(Claim.c.normalized == takeover['normalized'])
.values(
is_controlling=case(
[(Claim.c.claim_hash == takeover['claim_hash'], True)],
else_=False
),
takeover_height=case(
[(Claim.c.claim_hash == takeover['claim_hash'], takeover['height'])],
else_=None
),
activation_height=least(Claim.c.activation_height, takeover['height']),
)
)
result = c.execute(update_claims)
p.add(result.rowcount)
@event_emitter("blockchain.sync.claims.stakes", "claims")
def update_stakes(blocks: Tuple[int, int], claims: int, p: ProgressContext):
p.start(claims)
sql = (
Claim.update()
.where(where_claims_with_changed_supports(blocks))
.values(
staked_amount=(
Claim.c.amount +
claims_in_channel_amount_calc(Claim.c.claim_hash) +
staked_support_amount_calc(Claim.c.claim_hash)
),
staked_support_amount=staked_support_amount_calc(Claim.c.claim_hash),
staked_support_count=staked_support_count_calc(Claim.c.claim_hash),
)
)
result = p.ctx.execute(sql)
p.step(result.rowcount)
@event_emitter("blockchain.sync.claims.reposts", "claims")
def update_reposts(blocks: Tuple[int, int], claims: int, p: ProgressContext):
p.start(claims)
sql = (
Claim.update()
.where(where_claims_with_changed_reposts(blocks))
.values(reposted_count=reposted_claim_count_calc(Claim))
)
result = p.ctx.execute(sql)
p.step(result.rowcount)
@event_emitter("blockchain.sync.claims.channels", "channels")
def update_channel_stats(blocks: Tuple[int, int], initial_sync: int, p: ProgressContext):
update_sql = Claim.update().values(
signed_claim_count=channel_content_count_calc(Claim.alias('content')),
signed_support_count=channel_content_count_calc(Support),
)
if initial_sync:
p.start(p.ctx.fetchtotal(Claim.c.claim_type == TXO_TYPES['channel']), label="channel stats")
update_sql = update_sql.where(Claim.c.claim_type == TXO_TYPES['channel'])
elif blocks:
p.start(count_channels_with_changed_content(blocks), label="channel stats")
update_sql = update_sql.where(where_channels_with_changed_content(blocks))
else:
return
result = p.ctx.execute(update_sql)
if result.rowcount and p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM claim;"))
p.step(result.rowcount)
def select_reposts(channel_hashes, filter_type):
return (
select(Claim.c.reposted_claim_hash, filter_type, Claim.c.channel_hash).where(
(Claim.c.channel_hash.in_(channel_hashes)) &
(Claim.c.reposted_claim_hash.isnot(None))
)
)
@event_emitter("blockchain.sync.claims.filters", "claim_filters")
def update_claim_filters(resolve_censor_channel_hashes, search_censor_channel_hashes, p: ProgressContext):
p.ctx.execute(CensoredClaim.delete())
# order matters: first we insert the resolve filters; then the search ones.
# a claim that's censored in resolve is automatically also censored in search results.
p.ctx.execute(CensoredClaim.insert().from_select(
['claim_hash', 'censor_type', 'censoring_channel_hash'],
select_reposts(resolve_censor_channel_hashes, Censor.RESOLVE)
))
p.ctx.execute(p.ctx.insert_or_ignore(CensoredClaim).from_select(
['claim_hash', 'censor_type', 'censoring_channel_hash'],
select_reposts(search_censor_channel_hashes, Censor.SEARCH)
))
+25
View File
@@ -0,0 +1,25 @@
from contextvars import ContextVar
from lbry.db import query_context
from lbry.blockchain.lbrycrd import Lbrycrd
_chain: ContextVar[Lbrycrd] = ContextVar('chain')
def get_or_initialize_lbrycrd(ctx=None) -> Lbrycrd:
chain = _chain.get(None)
if chain is not None:
return chain
chain = Lbrycrd((ctx or query_context.context()).ledger)
chain.db.sync_open()
_chain.set(chain)
return chain
def uninitialize():
chain = _chain.get(None)
if chain is not None:
chain.db.sync_close()
chain.sync_run(chain.close_session())
_chain.set(None)
+79
View File
@@ -0,0 +1,79 @@
from typing import Dict
def split_range_into_10k_batches(start, end):
batch = [start, end]
batches = [batch]
for block in range(start, end+1):
if 0 < block != batch[0] and block % 10_000 == 0:
batch = [block, block]
batches.append(batch)
else:
batch[1] = block
return batches
class GroupFilter:
"""
Collects addresses into buckets of specific sizes defined by 10 raised to power of factor.
eg. a factor of 2 (10**2) would create block buckets 100-199, 200-299, etc
a factor of 3 (10**3) would create block buckets 1000-1999, 2000-2999, etc
"""
def __init__(self, start, end, factor):
self.start = start
self.end = end
self.factor = factor
self.resolution = resolution = 10**factor
last_height_in_group, groups = resolution-1, {}
for block in range(start, end+1):
if block % resolution == last_height_in_group:
groups[block-last_height_in_group] = set()
self.last_height_in_group = last_height_in_group
self.groups: Dict[int, set] = groups
@property
def coverage(self):
return list(self.groups.keys())
def add(self, height, addresses):
group = self.groups.get(height - (height % self.resolution))
if group is not None:
group.update(addresses)
class FilterBuilder:
"""
Creates filter groups, calculates the necessary block range to fulfill creation
of filter groups and collects tx filters, block filters and group filters.
"""
def __init__(self, start, end):
self.start = start
self.end = end
self.group_filters = [
GroupFilter(start, end, 4),
GroupFilter(start, end, 3),
GroupFilter(start, end, 2),
]
self.start_tx_height, self.end_tx_height = self._calculate_tx_heights_for_query()
self.tx_filters = []
self.block_filters: Dict[int, set] = {}
def _calculate_tx_heights_for_query(self):
for group_filter in self.group_filters:
if group_filter.groups:
return group_filter.coverage[0], self.end
return self.start, self.end
@property
def query_heights(self):
return self.start_tx_height, self.end_tx_height
def add(self, tx_hash, height, addresses):
if self.start <= height <= self.end:
self.tx_filters.append((tx_hash, height, addresses))
block_filter = self.block_filters.get(height)
if block_filter is None:
block_filter = self.block_filters[height] = set()
block_filter.update(addresses)
for group_filter in self.group_filters:
group_filter.add(height, addresses)
+95
View File
@@ -0,0 +1,95 @@
import logging
from typing import Tuple
from sqlalchemy import case, desc, text
from sqlalchemy.future import select
from lbry.db.tables import TX, TXO, Support, pg_add_support_constraints_and_indexes
from lbry.db.query_context import ProgressContext, event_emitter
from lbry.db.queries import row_to_txo
from lbry.db.constants import TXO_TYPES
from lbry.db.queries.txio import (
minimum_txo_columns,
where_unspent_txos, where_abandoned_supports,
count_unspent_txos,
)
from .claims import make_label
log = logging.getLogger(__name__)
@event_emitter("blockchain.sync.supports.insert", "supports")
def supports_insert(
blocks: Tuple[int, int],
missing_in_supports_table: bool,
flush_size: int,
p: ProgressContext
):
p.start(
count_unspent_txos(
TXO_TYPES['support'], blocks,
missing_in_supports_table=missing_in_supports_table,
), progress_id=blocks[0], label=make_label("add supprt", blocks)
)
channel_txo = TXO.alias('channel_txo')
select_supports = select(
*minimum_txo_columns, TXO.c.claim_hash,
TXO.c.signature, TXO.c.signature_digest,
case([(
TXO.c.channel_hash.isnot(None),
select(channel_txo.c.public_key).select_from(channel_txo).where(
(channel_txo.c.txo_type == TXO_TYPES['channel']) &
(channel_txo.c.claim_hash == TXO.c.channel_hash) &
(channel_txo.c.height <= TXO.c.height)
).order_by(desc(channel_txo.c.height)).limit(1).scalar_subquery()
)]).label('channel_public_key'),
).select_from(
TXO.join(TX)
).where(
where_unspent_txos(
TXO_TYPES['support'], blocks,
missing_in_supports_table=missing_in_supports_table,
)
)
with p.ctx.connect_streaming() as c:
loader = p.ctx.get_bulk_loader()
for row in c.execute(select_supports):
txo = row_to_txo(row)
loader.add_support(
txo,
signature=row.signature,
signature_digest=row.signature_digest,
channel_public_key=row.channel_public_key
)
if len(loader.supports) >= flush_size:
p.add(loader.flush(Support))
p.add(loader.flush(Support))
@event_emitter("blockchain.sync.supports.delete", "supports")
def supports_delete(supports, p: ProgressContext):
p.start(supports, label="del supprt")
deleted = p.ctx.execute(Support.delete().where(where_abandoned_supports()))
p.step(deleted.rowcount)
@event_emitter("blockchain.sync.supports.indexes", "steps")
def supports_constraints_and_indexes(p: ProgressContext):
p.start(1 + len(pg_add_support_constraints_and_indexes))
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM ANALYZE support;"))
p.step()
for constraint in pg_add_support_constraints_and_indexes:
if p.ctx.is_postgres:
p.ctx.execute(text(constraint))
p.step()
@event_emitter("blockchain.sync.supports.vacuum", "steps")
def supports_vacuum(p: ProgressContext):
p.start(1)
if p.ctx.is_postgres:
p.ctx.execute_notx(text("VACUUM support;"))
p.step()
+412
View File
@@ -0,0 +1,412 @@
import os
import asyncio
import logging
from binascii import unhexlify
from typing import Optional, Tuple, Set, List, Coroutine
from concurrent.futures import ThreadPoolExecutor
from lbry.db import Database, trending
from lbry.db import queries as q
from lbry.db.constants import TXO_TYPES, CLAIM_TYPE_CODES
from lbry.db.query_context import Event, Progress
from lbry.event import BroadcastSubscription, EventController
from lbry.service.base import Sync, BlockEvent
from lbry.blockchain.lbrycrd import Lbrycrd
from lbry.error import LbrycrdEventSubscriptionError
from . import blocks as block_phase, claims as claim_phase, supports as support_phase
from .context import uninitialize
from .filter_builder import split_range_into_10k_batches
log = logging.getLogger(__name__)
BLOCKS_INIT_EVENT = Event.add("blockchain.sync.blocks.init", "steps")
BLOCKS_MAIN_EVENT = Event.add("blockchain.sync.blocks.main", "blocks", "txs")
FILTER_INIT_EVENT = Event.add("blockchain.sync.filters.init", "steps")
FILTER_MAIN_EVENT = Event.add("blockchain.sync.filters.main", "blocks")
CLAIMS_INIT_EVENT = Event.add("blockchain.sync.claims.init", "steps")
CLAIMS_MAIN_EVENT = Event.add("blockchain.sync.claims.main", "claims")
TRENDS_INIT_EVENT = Event.add("blockchain.sync.trends.init", "steps")
TRENDS_MAIN_EVENT = Event.add("blockchain.sync.trends.main", "blocks")
SUPPORTS_INIT_EVENT = Event.add("blockchain.sync.supports.init", "steps")
SUPPORTS_MAIN_EVENT = Event.add("blockchain.sync.supports.main", "supports")
class BlockchainSync(Sync):
TX_FLUSH_SIZE = 25_000 # flush to db after processing this many TXs and update progress
CLAIM_FLUSH_SIZE = 25_000 # flush to db after processing this many claims and update progress
SUPPORT_FLUSH_SIZE = 25_000 # flush to db after processing this many supports and update progress
FILTER_FLUSH_SIZE = 10_000 # flush to db after processing this many filters and update progress
def __init__(self, chain: Lbrycrd, db: Database):
super().__init__(chain.ledger, db)
self.chain = chain
self.pid = os.getpid()
self._on_block_controller = EventController()
self.on_block = self._on_block_controller.stream
self.conf.events.register("blockchain.block", self.on_block)
self._on_mempool_controller = EventController()
self.on_mempool = self._on_mempool_controller.stream
self.on_block_hash_subscription: Optional[BroadcastSubscription] = None
self.on_tx_hash_subscription: Optional[BroadcastSubscription] = None
self.advance_loop_task: Optional[asyncio.Task] = None
self.block_hash_event = asyncio.Event()
self.tx_hash_event = asyncio.Event()
self.mempool = []
self.search_censor_channel_hashes = {
unhexlify(channel_id)[::-1] for channel_id in self.conf.search_censor_channel_ids
}
self.resolve_censor_channel_hashes = {
unhexlify(channel_id)[::-1] for channel_id in self.conf.resolve_censor_channel_ids
}
async def wait_for_chain_ready(self):
while True:
try:
return await self.chain.ensure_subscribable()
except asyncio.CancelledError:
raise
except LbrycrdEventSubscriptionError as e:
log.warning(
"Lbrycrd is misconfigured. Please double check if"
" zmqpubhashblock is properly set on lbrycrd.conf"
)
raise
except Exception as e:
log.warning("Blockchain not ready, waiting for it: %s", str(e))
await asyncio.sleep(1)
async def start(self):
self.db.stop_event.clear()
await self.wait_for_chain_ready()
self.advance_loop_task = asyncio.create_task(self.advance())
await self.advance_loop_task
await self.chain.subscribe()
self.advance_loop_task = asyncio.create_task(self.advance_loop())
self.on_block_hash_subscription = self.chain.on_block_hash.listen(
lambda e: self.block_hash_event.set()
)
self.on_tx_hash_subscription = self.chain.on_tx_hash.listen(
lambda e: self.tx_hash_event.set()
)
async def stop(self):
self.chain.unsubscribe()
self.db.stop_event.set()
for subscription in (
self.on_block_hash_subscription,
self.on_tx_hash_subscription,
self.advance_loop_task
):
if subscription is not None:
subscription.cancel()
if isinstance(self.db.executor, ThreadPoolExecutor):
await self.db.run(uninitialize)
async def run_tasks(self, tasks: List[Coroutine]) -> Optional[Set[asyncio.Future]]:
done, pending = await asyncio.wait(
tasks, return_when=asyncio.FIRST_EXCEPTION
)
if pending:
self.db.stop_event.set()
for future in pending:
future.cancel()
for future in done:
future.result()
return
return done
async def get_block_headers(self, start_height: int, end_height: int = None):
return await self.db.get_block_headers(start_height, end_height)
async def get_best_block_height(self) -> int:
return await self.db.get_best_block_height()
async def get_best_block_height_for_file(self, file_number) -> int:
return await self.db.run(
block_phase.get_best_block_height_for_file, file_number
)
async def sync_blocks(self) -> Optional[Tuple[int, int]]:
tasks = []
starting_height = None
tx_count = block_count = 0
with Progress(self.db.message_queue, BLOCKS_INIT_EVENT) as p:
ending_height = await self.chain.db.get_best_height()
for chain_file in p.iter(await self.chain.db.get_block_files()):
# block files may be read and saved out of order, need to check
# each file individually to see if we have missing blocks
our_best_file_height = await self.get_best_block_height_for_file(
chain_file['file_number']
)
if our_best_file_height == chain_file['best_height']:
# we have all blocks in this file, skipping
continue
if -1 < our_best_file_height < chain_file['best_height']:
# we have some blocks, need to figure out what we're missing
# call get_block_files again limited to this file and current_height
chain_file = (await self.chain.db.get_block_files(
file_number=chain_file['file_number'], start_height=our_best_file_height+1,
))[0]
tx_count += chain_file['txs']
block_count += chain_file['blocks']
file_start_height = chain_file['start_height']
starting_height = min(
file_start_height if starting_height is None else starting_height,
file_start_height
)
tasks.append(self.db.run(
block_phase.sync_block_file, chain_file['file_number'], file_start_height,
chain_file['txs'], self.TX_FLUSH_SIZE
))
with Progress(self.db.message_queue, BLOCKS_MAIN_EVENT) as p:
p.start(block_count, tx_count, extra={
"starting_height": starting_height,
"ending_height": ending_height,
"files": len(tasks),
"claims": await self.chain.db.get_claim_metadata_count(starting_height, ending_height),
"supports": await self.chain.db.get_support_metadata_count(starting_height, ending_height),
})
completed = await self.run_tasks(tasks)
if completed:
if starting_height == 0:
await self.db.run(block_phase.blocks_constraints_and_indexes)
else:
await self.db.run(block_phase.blocks_vacuum)
best_height_processed = max(f.result() for f in completed)
return starting_height, best_height_processed
async def sync_filters(self):
with Progress(self.db.message_queue, FILTER_INIT_EVENT) as p:
p.start(2)
initial_sync = not await self.db.has_filters()
p.step()
if initial_sync:
blocks = [0, await self.db.get_best_block_height()]
else:
blocks = await self.db.run(block_phase.get_block_range_without_filters)
if blocks != (-1, -1):
batches = split_range_into_10k_batches(*blocks)
p.step()
else:
p.step()
return
with Progress(self.db.message_queue, FILTER_MAIN_EVENT) as p:
p.start((blocks[1]-blocks[0])+1)
await self.run_tasks([
self.db.run(block_phase.sync_filters, *batch) for batch in batches
])
if initial_sync:
await self.db.run(block_phase.filters_constraints_and_indexes)
else:
await self.db.run(block_phase.filters_vacuum)
async def sync_spends(self, blocks_added):
if blocks_added:
await self.db.run(block_phase.sync_spends, blocks_added[0] == 0)
async def count_unspent_txos(
self,
txo_types: Tuple[int, ...],
blocks: Tuple[int, int] = None,
missing_in_supports_table: bool = False,
missing_in_claims_table: bool = False,
missing_or_stale_in_claims_table: bool = False,
) -> int:
return await self.db.run(
q.count_unspent_txos, txo_types, blocks,
missing_in_supports_table,
missing_in_claims_table,
missing_or_stale_in_claims_table,
)
async def distribute_unspent_txos(
self,
txo_types: Tuple[int, ...],
blocks: Tuple[int, int] = None,
missing_in_supports_table: bool = False,
missing_in_claims_table: bool = False,
missing_or_stale_in_claims_table: bool = False,
) -> int:
return await self.db.run(
q.distribute_unspent_txos, txo_types, blocks,
missing_in_supports_table,
missing_in_claims_table,
missing_or_stale_in_claims_table,
self.db.workers
)
async def count_abandoned_supports(self) -> int:
return await self.db.run(q.count_abandoned_supports)
async def count_abandoned_claims(self) -> int:
return await self.db.run(q.count_abandoned_claims)
async def count_claims_with_changed_supports(self, blocks) -> int:
return await self.db.run(q.count_claims_with_changed_supports, blocks)
async def count_claims_with_changed_reposts(self, blocks) -> int:
return await self.db.run(q.count_claims_with_changed_reposts, blocks)
async def count_channels_with_changed_content(self, blocks) -> int:
return await self.db.run(q.count_channels_with_changed_content, blocks)
async def count_takeovers(self, blocks) -> int:
return await self.chain.db.get_takeover_count(
start_height=blocks[0], end_height=blocks[-1]
)
async def sync_claims(self, blocks) -> bool:
delete_claims = takeovers = claims_with_changed_supports = claims_with_changed_reposts = 0
initial_sync = not await self.db.has_filters()
with Progress(self.db.message_queue, CLAIMS_INIT_EVENT) as p:
if initial_sync:
total, batches = await self.distribute_unspent_txos(CLAIM_TYPE_CODES)
elif blocks:
p.start(5)
# 1. content claims to be inserted or updated
total = await self.count_unspent_txos(
CLAIM_TYPE_CODES, blocks, missing_or_stale_in_claims_table=True
)
batches = [blocks] if total else []
p.step()
# 2. claims to be deleted
delete_claims = await self.count_abandoned_claims()
total += delete_claims
p.step()
# 3. claims to be updated with new support totals
claims_with_changed_supports = await self.count_claims_with_changed_supports(blocks)
total += claims_with_changed_supports
p.step()
# 4. claims to be updated with new repost totals
claims_with_changed_reposts = await self.count_claims_with_changed_reposts(blocks)
total += claims_with_changed_reposts
p.step()
# 5. claims to be updated due to name takeovers
takeovers = await self.count_takeovers(blocks)
total += takeovers
p.step()
else:
return initial_sync
with Progress(self.db.message_queue, CLAIMS_MAIN_EVENT) as p:
p.start(total)
if batches:
await self.run_tasks([
self.db.run(claim_phase.claims_insert, batch, not initial_sync, self.CLAIM_FLUSH_SIZE)
for batch in batches
])
if not initial_sync:
await self.run_tasks([
self.db.run(claim_phase.claims_update, batch) for batch in batches
])
if delete_claims:
await self.db.run(claim_phase.claims_delete, delete_claims)
if takeovers:
await self.db.run(claim_phase.update_takeovers, blocks, takeovers)
if claims_with_changed_supports:
await self.db.run(claim_phase.update_stakes, blocks, claims_with_changed_supports)
if claims_with_changed_reposts:
await self.db.run(claim_phase.update_reposts, blocks, claims_with_changed_reposts)
if initial_sync:
await self.db.run(claim_phase.claims_constraints_and_indexes)
else:
await self.db.run(claim_phase.claims_vacuum)
return initial_sync
async def sync_supports(self, blocks):
delete_supports = 0
initial_sync = not await self.db.has_supports()
with Progress(self.db.message_queue, SUPPORTS_INIT_EVENT) as p:
if initial_sync:
total, support_batches = await self.distribute_unspent_txos(TXO_TYPES['support'])
elif blocks:
p.start(2)
# 1. supports to be inserted
total = await self.count_unspent_txos(
TXO_TYPES['support'], blocks, missing_in_supports_table=True
)
support_batches = [blocks] if total else []
p.step()
# 2. supports to be deleted
delete_supports = await self.count_abandoned_supports()
total += delete_supports
p.step()
else:
return
with Progress(self.db.message_queue, SUPPORTS_MAIN_EVENT) as p:
p.start(total)
if support_batches:
await self.run_tasks([
self.db.run(
support_phase.supports_insert, batch, not initial_sync, self.SUPPORT_FLUSH_SIZE
) for batch in support_batches
])
if delete_supports:
await self.db.run(support_phase.supports_delete, delete_supports)
if initial_sync:
await self.db.run(support_phase.supports_constraints_and_indexes)
else:
await self.db.run(support_phase.supports_vacuum)
async def sync_channel_stats(self, blocks, initial_sync):
await self.db.run(claim_phase.update_channel_stats, blocks, initial_sync)
async def sync_trends(self):
ending_height = await self.chain.db.get_best_height()
if ending_height is not None:
await self.db.run(trending.calculate_trending, ending_height)
async def sync_claim_filtering(self):
await self.db.run(
claim_phase.update_claim_filters,
self.resolve_censor_channel_hashes,
self.search_censor_channel_hashes
)
async def advance(self):
blocks_added = await self.sync_blocks()
await self.sync_spends(blocks_added)
await self.sync_filters()
initial_claim_sync = await self.sync_claims(blocks_added)
await self.sync_supports(blocks_added)
await self.sync_channel_stats(blocks_added, initial_claim_sync)
await self.sync_trends()
await self.sync_claim_filtering()
if blocks_added:
await self._on_block_controller.add(BlockEvent(blocks_added[-1]))
async def sync_mempool(self):
added = await self.db.run(block_phase.sync_mempool)
await self.sync_spends([-1])
await self.db.run(claim_phase.claims_insert, [-1, -1], True, self.CLAIM_FLUSH_SIZE)
await self.db.run(claim_phase.claims_update, [-1, -1])
await self.db.run(claim_phase.claims_vacuum)
self.mempool.extend(added)
await self._on_mempool_controller.add(added)
async def clear_mempool(self):
self.mempool.clear()
await self.db.run(block_phase.clear_mempool)
async def advance_loop(self):
while True:
try:
await asyncio.wait([
self.tx_hash_event.wait(),
self.block_hash_event.wait(),
], return_when=asyncio.FIRST_COMPLETED)
if self.block_hash_event.is_set():
self.block_hash_event.clear()
await self.clear_mempool()
await self.advance()
self.tx_hash_event.clear()
await self.sync_mempool()
except asyncio.CancelledError:
return
except Exception as e:
log.exception(e)
await self.stop()
async def rewind(self, height):
await self.db.run(block_phase.rewind, height)
+837
View File
@@ -0,0 +1,837 @@
import struct
import hashlib
import logging
import asyncio
from datetime import date
from binascii import hexlify, unhexlify
from typing import List, Iterable, Optional, Union
import ecdsa
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.serialization import load_der_public_key
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives.asymmetric.utils import Prehashed
from cryptography.exceptions import InvalidSignature
from lbry.crypto.hash import hash160, sha256
from lbry.crypto.base58 import Base58
from lbry.schema.url import normalize_name
from lbry.schema.claim import Claim
from lbry.schema.base import Signable
from lbry.schema.purchase import Purchase
from lbry.schema.support import Support
from .script import InputScript, OutputScript
from .bcd_data_stream import BCDataStream
from .hash import TXRef, TXRefImmutable
from .util import ReadOnlyList
log = logging.getLogger()
class TXRefMutable(TXRef):
__slots__ = ('tx',)
def __init__(self, tx: 'Transaction') -> None:
super().__init__()
self.tx = tx
@property
def id(self):
if self._id is None:
self._id = hexlify(self.hash[::-1]).decode()
return self._id
@property
def hash(self):
if self._hash is None:
self._hash = sha256(sha256(self.tx.raw_sans_segwit))
return self._hash
@property
def height(self):
return self.tx.height
@property
def timestamp(self):
return self.tx.timestamp
def reset(self):
self._id = None
self._hash = None
class TXORef:
__slots__ = 'tx_ref', 'position'
def __init__(self, tx_ref: TXRef, position: int) -> None:
self.tx_ref = tx_ref
self.position = position
@property
def id(self):
return f'{self.tx_ref.id}:{self.position}'
@property
def hash(self):
return self.tx_ref.hash + BCDataStream.uint32.pack(self.position)
@property
def is_null(self):
return self.tx_ref.is_null
@property
def txo(self) -> Optional['Output']:
return None
class TXORefResolvable(TXORef):
__slots__ = ('_txo',)
def __init__(self, txo: 'Output') -> None:
assert txo.tx_ref is not None
assert txo.position is not None
super().__init__(txo.tx_ref, txo.position)
self._txo = txo
@property
def txo(self):
return self._txo
class InputOutput:
__slots__ = 'tx_ref', 'position'
def __init__(self, tx_ref: Union[TXRef, TXRefImmutable] = None, position: int = None) -> None:
self.tx_ref = tx_ref
self.position = position
@property
def size(self) -> int:
""" Size of this input / output in bytes. """
stream = BCDataStream()
self.serialize_to(stream)
return len(stream.get_bytes())
def get_fee(self, ledger):
return self.size * ledger.fee_per_byte
def serialize_to(self, stream, alternate_script=None):
raise NotImplementedError
class Input(InputOutput):
NULL_SIGNATURE = b'\x00'*72
NULL_PUBLIC_KEY = b'\x00'*33
NULL_HASH32 = b'\x00'*32
__slots__ = 'txo_ref', 'sequence', 'coinbase', 'script'
def __init__(self, txo_ref: TXORef, script: InputScript, sequence: int = 0xFFFFFFFF,
tx_ref: TXRef = None, position: int = None) -> None:
super().__init__(tx_ref, position)
self.txo_ref = txo_ref
self.sequence = sequence
self.coinbase = script if txo_ref.is_null else None
self.script = script if not txo_ref.is_null else None
@property
def is_coinbase(self):
return self.coinbase is not None
@classmethod
def spend(cls, txo: 'Output') -> 'Input':
""" Create an input to spend the output."""
assert txo.script.is_pay_pubkey_hash, 'Attempting to spend unsupported output.'
script = InputScript.redeem_pubkey_hash(cls.NULL_SIGNATURE, cls.NULL_PUBLIC_KEY)
return cls(txo.ref, script)
@classmethod
def create_coinbase(cls) -> 'Input':
tx_ref = TXRefImmutable.from_hash(cls.NULL_HASH32, 0, 0)
txo_ref = TXORef(tx_ref, 0)
return cls(txo_ref, b'beef')
@property
def amount(self) -> int:
""" Amount this input adds to the transaction. """
if self.txo_ref.txo is None:
raise ValueError('Cannot resolve output to get amount.')
return self.txo_ref.txo.amount
@property
def is_my_input(self) -> Optional[bool]:
""" True if the output this input spends is yours. """
if self.txo_ref.txo is None:
return False
return self.txo_ref.txo.is_my_output
@classmethod
def deserialize_from(cls, stream):
tx_ref = TXRefImmutable.from_hash(stream.read(32), -1, -1)
position = stream.read_uint32()
script = stream.read_string()
sequence = stream.read_uint32()
return cls(
TXORef(tx_ref, position),
InputScript(script) if not tx_ref.is_null else script,
sequence
)
def serialize_to(self, stream, alternate_script=None):
stream.write(self.txo_ref.tx_ref.hash)
stream.write_uint32(self.txo_ref.position)
if alternate_script is not None:
stream.write_string(alternate_script)
else:
if self.is_coinbase:
stream.write_string(self.coinbase)
else:
stream.write_string(self.script.source)
stream.write_uint32(self.sequence)
class Output(InputOutput):
__slots__ = (
'amount', 'script', 'is_internal_transfer', 'spent_height', 'is_my_output', 'is_my_input',
'channel', 'private_key', 'meta', 'sent_supports', 'sent_tips', 'received_tips',
'purchase', 'purchased_claim', 'purchase_receipt',
'reposted_claim', 'claims', '_signable'
)
def __init__(self, amount: int, script: OutputScript,
tx_ref: TXRef = None, position: int = None,
is_internal_transfer: Optional[bool] = None, spent_height: Optional[bool] = None,
is_my_output: Optional[bool] = None, is_my_input: Optional[bool] = None,
sent_supports: Optional[int] = None, sent_tips: Optional[int] = None,
received_tips: Optional[int] = None,
channel: Optional['Output'] = None, private_key: Optional[str] = None
) -> None:
super().__init__(tx_ref, position)
self.amount = amount
self.script = script
self.is_internal_transfer = is_internal_transfer
self.spent_height = spent_height
self.is_my_output = is_my_output
self.is_my_input = is_my_input
self.sent_supports = sent_supports
self.sent_tips = sent_tips
self.received_tips = received_tips
self.channel = channel
self.private_key = private_key
self.purchase: 'Output' = None # txo containing purchase metadata
self.purchased_claim: 'Output' = None # resolved claim pointed to by purchase
self.purchase_receipt: 'Output' = None # txo representing purchase receipt for this claim
self.reposted_claim: 'Output' = None # txo representing claim being reposted
self.claims: List['Output'] = None # resolved claims for collection
self._signable: Optional[Signable] = None
self.meta = {}
def update_annotations(self, annotated: 'Output'):
if annotated is None:
self.is_internal_transfer = None
self.spent_height = None
self.is_my_output = None
self.is_my_input = None
self.sent_supports = None
self.sent_tips = None
self.received_tips = None
else:
self.is_internal_transfer = annotated.is_internal_transfer
self.spent_height = annotated.spent_height
self.is_my_output = annotated.is_my_output
self.is_my_input = annotated.is_my_input
self.sent_supports = annotated.sent_supports
self.sent_tips = annotated.sent_tips
self.received_tips = annotated.received_tips
self.channel = annotated.channel if annotated else None
self.private_key = annotated.private_key if annotated else None
@property
def ref(self):
return TXORefResolvable(self)
@property
def id(self):
return self.ref.id
@property
def hash(self):
return self.ref.hash
@property
def is_spent(self):
if self.spent_height is not None:
return self.spent_height > 0
@property
def pubkey_hash(self):
pubkey_hash = self.script.values.get('pubkey_hash')
if pubkey_hash:
return pubkey_hash
return hash160(self.script.values['pubkey'])
@property
def has_address(self):
return (
'pubkey_hash' in self.script.values or
'script_hash' in self.script.values or
'pubkey' in self.script.values
)
def get_address(self, ledger):
if 'script_hash' in self.script.values:
return ledger.script_hash_to_address(self.script.values['script_hash'])
return ledger.pubkey_hash_to_address(self.pubkey_hash)
@classmethod
def pay_pubkey_hash(cls, amount, pubkey_hash):
return cls(amount, OutputScript.pay_pubkey_hash(pubkey_hash))
@classmethod
def deserialize_from(cls, stream, transaction_offset: int = 0):
amount = stream.read_uint64()
length = stream.read_compact_size()
offset = stream.tell()-transaction_offset
script = OutputScript(stream.read(length), offset=offset)
return cls(amount=amount, script=script)
def serialize_to(self, stream, alternate_script=None):
stream.write_uint64(self.amount)
stream.write_string(self.script.source)
def get_fee(self, ledger):
name_fee = 0
if self.script.is_claim_name:
name_fee = len(self.script.values['claim_name']) * ledger.fee_per_name_char
return max(name_fee, super().get_fee(ledger))
@property
def is_claim(self) -> bool:
return self.script.is_claim_name or self.script.is_update_claim
@property
def is_support(self) -> bool:
return self.script.is_support_claim
@property
def is_support_data(self) -> bool:
return self.script.is_support_claim_data
@property
def claim_hash(self) -> bytes:
if self.script.is_claim_name:
return hash160(self.tx_ref.hash + struct.pack('>I', self.position))
elif self.script.is_update_claim or self.script.is_support_claim:
return self.script.values['claim_id']
else:
raise ValueError('No claim_id associated.')
@property
def claim_id(self) -> str:
return hexlify(self.claim_hash[::-1]).decode()
@property
def claim_name(self) -> str:
if self.script.is_claim_involved:
return self.script.values['claim_name'].decode()
raise ValueError('No claim_name associated.')
@property
def normalized_name(self) -> str:
return normalize_name(self.claim_name)
@property
def claim(self) -> Claim:
if self.is_claim:
if not isinstance(self.script.values['claim'], Claim):
self.script.values['claim'] = Claim.from_bytes(self.script.values['claim'])
return self.script.values['claim']
raise ValueError('Only claim name and claim update have the claim payload.')
@property
def can_decode_claim(self):
try:
return self.claim
except Exception:
return False
@property
def support(self) -> Support:
if self.is_support_data:
if not isinstance(self.script.values['support'], Support):
self.script.values['support'] = Support.from_bytes(self.script.values['support'])
return self.script.values['support']
raise ValueError('Only supports with data can be represented as Supports.')
@property
def can_decode_support(self):
try:
return self.support
except Exception:
return False
@property
def signable(self) -> Signable:
if self._signable is None:
if self.is_claim:
self._signable = self.claim
elif self.is_support_data:
self._signable = self.support
return self._signable
@property
def can_decode_signable(self) -> Signable:
try:
return self.signable
except Exception:
return False
@property
def permanent_url(self) -> str:
if self.script.is_claim_involved:
return f"lbry://{self.claim_name}#{self.claim_id}"
raise ValueError('No claim associated.')
@property
def has_private_key(self):
return self.private_key is not None
def get_signature_digest(self, ledger):
if self.signable.unsigned_payload:
pieces = [
Base58.decode(self.get_address(ledger)),
self.signable.unsigned_payload,
self.signable.signing_channel_hash[::-1]
]
else:
pieces = [
self.tx_ref.tx.inputs[0].txo_ref.hash,
self.signable.signing_channel_hash,
self.signable.to_message_bytes()
]
return sha256(b''.join(pieces))
def get_encoded_signature(self):
signature = hexlify(self.signable.signature)
r = int(signature[:int(len(signature)/2)], 16)
s = int(signature[int(len(signature)/2):], 16)
return ecdsa.util.sigencode_der(r, s, len(signature)*4)
@staticmethod
def is_signature_valid(encoded_signature, signature_digest, public_key_bytes):
try:
public_key = load_der_public_key(public_key_bytes, default_backend())
public_key.verify( # pylint: disable=no-value-for-parameter
encoded_signature, signature_digest,
ec.ECDSA(Prehashed(hashes.SHA256()))
)
return True
except (ValueError, InvalidSignature):
pass
return False
def is_signed_by(self, channel: 'Output', ledger=None):
return self.is_signature_valid(
self.get_encoded_signature(),
self.get_signature_digest(ledger),
channel.claim.channel.public_key_bytes
)
def sign(self, channel: 'Output', first_input_id=None):
self.channel = channel
self.signable.signing_channel_hash = channel.claim_hash
digest = sha256(b''.join([
first_input_id or self.tx_ref.tx.inputs[0].txo_ref.hash,
self.signable.signing_channel_hash,
self.signable.to_message_bytes()
]))
self.signable.signature = channel.private_key.sign_digest_deterministic(digest, hashfunc=hashlib.sha256)
self.script.generate()
def clear_signature(self):
self.channel = None
self.claim.clear_signature()
@staticmethod
def _sync_generate_channel_private_key():
private_key = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1, hashfunc=hashlib.sha256)
public_key_bytes = private_key.get_verifying_key().to_der()
return private_key, public_key_bytes
async def generate_channel_private_key(self):
private_key, public_key_bytes = await asyncio.get_running_loop().run_in_executor(
None, Output._sync_generate_channel_private_key
)
self.private_key = private_key
self.claim.channel.public_key_bytes = public_key_bytes
self.script.generate()
return self.private_key
def is_channel_private_key(self, private_key):
return self.claim.channel.public_key_bytes == private_key.get_verifying_key().to_der()
@classmethod
def pay_claim_name_pubkey_hash(
cls, amount: int, claim_name: str, claim: Claim, pubkey_hash: bytes) -> 'Output':
script = OutputScript.pay_claim_name_pubkey_hash(
claim_name.encode(), claim, pubkey_hash)
return cls(amount, script)
@classmethod
def pay_update_claim_pubkey_hash(
cls, amount: int, claim_name: str, claim_id: str, claim: Claim, pubkey_hash: bytes) -> 'Output':
script = OutputScript.pay_update_claim_pubkey_hash(
claim_name.encode(), unhexlify(claim_id)[::-1], claim, pubkey_hash
)
return cls(amount, script)
@classmethod
def pay_support_pubkey_hash(cls, amount: int, claim_name: str, claim_id: str, pubkey_hash: bytes) -> 'Output':
script = OutputScript.pay_support_pubkey_hash(
claim_name.encode(), unhexlify(claim_id)[::-1], pubkey_hash
)
return cls(amount, script)
@classmethod
def pay_support_data_pubkey_hash(
cls, amount: int, claim_name: str, claim_id: str, support: Support, pubkey_hash: bytes) -> 'Output':
script = OutputScript.pay_support_data_pubkey_hash(
claim_name.encode(), unhexlify(claim_id)[::-1], support, pubkey_hash
)
return cls(amount, script)
@classmethod
def add_purchase_data(cls, purchase: Purchase) -> 'Output':
script = OutputScript.return_data(purchase)
return cls(0, script)
@property
def is_purchase_data(self) -> bool:
return self.script.is_return_data and (
isinstance(self.script.values['data'], Purchase) or
Purchase.has_start_byte(self.script.values['data'])
)
@property
def purchase_data(self) -> Purchase:
if self.is_purchase_data:
if not isinstance(self.script.values['data'], Purchase):
self.script.values['data'] = Purchase.from_bytes(self.script.values['data'])
return self.script.values['data']
raise ValueError('Output does not have purchase data.')
@property
def can_decode_purchase_data(self):
try:
return self.purchase_data
except: # pylint: disable=bare-except
return False
@property
def purchased_claim_id(self):
if self.purchase is not None:
return self.purchase.purchase_data.claim_id
if self.purchased_claim is not None:
return self.purchased_claim.claim_id
@property
def purchased_claim_hash(self):
if self.purchase is not None:
return self.purchase.purchase_data.claim_hash
if self.purchased_claim is not None:
return self.purchased_claim.claim_hash
@property
def has_price(self):
if self.can_decode_claim:
claim = self.claim
if claim.is_stream:
stream = claim.stream
return stream.has_fee and stream.fee.amount and stream.fee.amount > 0
return False
@property
def price(self):
return self.claim.stream.fee
class Transaction:
def __init__(self, raw=None, version: int = 1, locktime: int = 0, is_verified: bool = False,
height: int = -2, position: int = -1, timestamp: int = 0) -> None:
self._raw = raw
self._raw_sans_segwit = None
self.is_segwit_flag = 0
self.witnesses: List[bytes] = []
self.ref = TXRefMutable(self)
self.version = version
self.locktime = locktime
self._inputs: List[Input] = []
self._outputs: List[Output] = []
self.is_verified = is_verified
# Height Progression
# -2: not broadcast
# -1: in mempool but has unconfirmed inputs
# 0: in mempool and all inputs confirmed
# +num: confirmed in a specific block (height)
self.height = height
self.position = position
self.timestamp = timestamp
self._day: int = 0
if raw is not None:
self.deserialize()
def __repr__(self):
return f"TX({self.id[:10]}...{self.id[-10:]})"
@property
def is_broadcast(self):
return self.height > -2
@property
def is_mempool(self):
return self.height in (-1, 0)
@property
def is_confirmed(self):
return self.height > 0
@property
def id(self):
return self.ref.id
@property
def hash(self):
return self.ref.hash
@property
def day(self):
if self._day is None and self.timestamp > 0:
self._day = date.fromtimestamp(self.timestamp).toordinal()
return self._day
@property
def raw(self):
if self._raw is None:
self._raw = self._serialize()
return self._raw
@property
def raw_sans_segwit(self):
if self.is_segwit_flag:
if self._raw_sans_segwit is None:
self._raw_sans_segwit = self._serialize(sans_segwit=True)
return self._raw_sans_segwit
return self.raw
def _reset(self):
self._raw = None
self._raw_sans_segwit = None
self.ref.reset()
@property
def inputs(self) -> ReadOnlyList[Input]:
return ReadOnlyList(self._inputs)
@property
def outputs(self) -> ReadOnlyList[Output]:
return ReadOnlyList(self._outputs)
def _add(self, existing_ios: List, new_ios: Iterable[InputOutput], reset=False) -> 'Transaction':
for txio in new_ios:
txio.tx_ref = self.ref
txio.position = len(existing_ios)
existing_ios.append(txio)
if reset:
self._reset()
return self
def add_inputs(self, inputs: Iterable[Input]) -> 'Transaction':
return self._add(self._inputs, inputs, True)
def add_outputs(self, outputs: Iterable[Output]) -> 'Transaction':
return self._add(self._outputs, outputs, True)
@property
def size(self) -> int:
""" Size in bytes of the entire transaction. """
return len(self.raw)
@property
def base_size(self) -> int:
""" Size of transaction without inputs or outputs in bytes. """
return (
self.size
- sum(txi.size for txi in self._inputs)
- sum(txo.size for txo in self._outputs)
)
@property
def input_sum(self):
return sum(i.amount for i in self.inputs if i.txo_ref.txo is not None)
@property
def output_sum(self):
return sum(o.amount for o in self.outputs)
@property
def net_account_balance(self) -> int:
balance = 0
for txi in self.inputs:
if txi.txo_ref.txo is None:
continue
if txi.is_my_input is True:
balance -= txi.amount
elif txi.is_my_input is None:
raise ValueError(
"Cannot access net_account_balance if inputs do not "
"have is_my_input set properly."
)
for txo in self.outputs:
if txo.is_my_output is True:
balance += txo.amount
elif txo.is_my_output is None:
raise ValueError(
"Cannot access net_account_balance if outputs do not "
"have is_my_output set properly."
)
return balance
@property
def fee(self) -> int:
return self.input_sum - self.output_sum
def get_base_fee(self, ledger) -> int:
""" Fee for base tx excluding inputs and outputs. """
return self.base_size * ledger.fee_per_byte
def get_effective_input_sum(self, ledger) -> int:
""" Sum of input values *minus* the cost involved to spend them. """
return sum(txi.amount - txi.get_fee(ledger) for txi in self._inputs)
def get_total_output_sum(self, ledger) -> int:
""" Sum of output values *plus* the cost involved to spend them. """
return sum(txo.amount + txo.get_fee(ledger) for txo in self._outputs)
def _serialize(self, with_inputs: bool = True, sans_segwit: bool = False) -> bytes:
stream = BCDataStream()
stream.write_uint32(self.version)
if with_inputs:
stream.write_compact_size(len(self._inputs))
for txin in self._inputs:
txin.serialize_to(stream)
stream.write_compact_size(len(self._outputs))
for txout in self._outputs:
txout.serialize_to(stream)
stream.write_uint32(self.locktime)
return stream.get_bytes()
def _serialize_for_signature(self, signing_input: int) -> bytes:
stream = BCDataStream()
stream.write_uint32(self.version)
stream.write_compact_size(len(self._inputs))
for i, txin in enumerate(self._inputs):
if signing_input == i:
assert txin.txo_ref.txo is not None
txin.serialize_to(stream, txin.txo_ref.txo.script.source)
else:
txin.serialize_to(stream, b'')
stream.write_compact_size(len(self._outputs))
for txout in self._outputs:
txout.serialize_to(stream)
stream.write_uint32(self.locktime)
stream.write_uint32(self.signature_hash_type(1)) # signature hash type: SIGHASH_ALL
return stream.get_bytes()
def deserialize(self, stream=None):
if self._raw is not None or stream is not None:
stream = stream or BCDataStream(self._raw)
start = stream.tell()
self.version = stream.read_uint32()
input_count = stream.read_compact_size()
if input_count == 0:
self.is_segwit_flag = stream.read_uint8()
input_count = stream.read_compact_size()
self._add(self._inputs, [
Input.deserialize_from(stream) for _ in range(input_count)
])
output_count = stream.read_compact_size()
self._add(self._outputs, [
Output.deserialize_from(stream, start) for _ in range(output_count)
])
if self.is_segwit_flag:
# drain witness portion of transaction
# too many witnesses for no crime
self.witnesses = []
for _ in range(input_count):
for _ in range(stream.read_compact_size()):
self.witnesses.append(stream.read(stream.read_compact_size()))
self.locktime = stream.read_uint32()
return self
@staticmethod
def signature_hash_type(hash_type):
return hash_type
@property
def my_inputs(self):
for txi in self.inputs:
if txi.txo_ref.txo is not None and txi.txo_ref.txo.is_my_output:
yield txi
def _filter_my_outputs(self, f):
for txo in self.outputs:
if txo.is_my_output and f(txo.script):
yield txo
def _filter_other_outputs(self, f):
for txo in self.outputs:
if not txo.is_my_output and f(txo.script):
yield txo
def _filter_any_outputs(self, f):
for txo in self.outputs:
if f(txo):
yield txo
@property
def my_claim_outputs(self):
return self._filter_my_outputs(lambda s: s.is_claim_name)
@property
def my_update_outputs(self):
return self._filter_my_outputs(lambda s: s.is_update_claim)
@property
def my_support_outputs(self):
return self._filter_my_outputs(lambda s: s.is_support_claim)
@property
def any_purchase_outputs(self):
return self._filter_any_outputs(lambda o: o.purchase is not None)
@property
def other_support_outputs(self):
return self._filter_other_outputs(lambda s: s.is_support_claim)
@property
def my_abandon_outputs(self):
for txi in self.inputs:
abandon = txi.txo_ref.txo
if abandon is not None and abandon.is_my_output and abandon.script.is_claim_involved:
is_update = False
if abandon.script.is_claim_name or abandon.script.is_update_claim:
for update in self.my_update_outputs:
if abandon.claim_id == update.claim_id:
is_update = True
break
if not is_update:
yield abandon
+93
View File
@@ -0,0 +1,93 @@
from typing import TypeVar, Sequence, Optional
T = TypeVar('T')
class ReadOnlyList(Sequence[T]):
def __init__(self, lst):
self.lst = lst
def __getitem__(self, key):
return self.lst[key]
def __len__(self) -> int:
return len(self.lst)
class ArithUint256:
# https://github.com/bitcoin/bitcoin/blob/master/src/arith_uint256.cpp
__slots__ = '_value', '_compact'
def __init__(self, value: int) -> None:
self._value = value
self._compact: Optional[int] = None
@classmethod
def from_compact(cls, compact) -> 'ArithUint256':
size = compact >> 24
word = compact & 0x007fffff
if size <= 3:
return cls(word >> 8 * (3 - size))
else:
return cls(word << 8 * (size - 3))
@property
def value(self) -> int:
return self._value
@property
def compact(self) -> int:
if self._compact is None:
self._compact = self._calculate_compact()
return self._compact
@property
def negative(self) -> int:
return self._calculate_compact(negative=True)
@property
def bits(self) -> int:
""" Returns the position of the highest bit set plus one. """
bits = bin(self._value)[2:]
for i, d in enumerate(bits):
if d:
return (len(bits) - i) + 1
return 0
@property
def low64(self) -> int:
return self._value & 0xffffffffffffffff
def _calculate_compact(self, negative=False) -> int:
size = (self.bits + 7) // 8
if size <= 3:
compact = self.low64 << 8 * (3 - size)
else:
compact = ArithUint256(self._value >> 8 * (size - 3)).low64
# The 0x00800000 bit denotes the sign.
# Thus, if it is already set, divide the mantissa by 256 and increase the exponent.
if compact & 0x00800000:
compact >>= 8
size += 1
assert (compact & ~0x007fffff) == 0
assert size < 256
compact |= size << 24
if negative and compact & 0x007fffff:
compact |= 0x00800000
return compact
def __mul__(self, x):
# Take the mod because we are limited to an unsigned 256 bit number
return ArithUint256((self._value * x) % 2 ** 256)
def __truediv__(self, x):
return ArithUint256(int(self._value / x))
def __gt__(self, other):
return self._value > other
def __lt__(self, other):
return self._value < other
@@ -1,2 +1,4 @@
# don't touch this. CI server changes this during build/deployment
BUILD = "dev"
COMMIT_HASH = "none"
DOCKER_TAG = "none"
+281
View File
@@ -0,0 +1,281 @@
import os
import sys
import asyncio
import pathlib
import argparse
import textwrap
import subprocess
from docopt import docopt
from lbry import __version__
from lbry.conf import Config, CLIConfig
from lbry.service import Daemon, Client, FullNode, LightClient
from lbry.service.metadata import interface
def split_subparser_argument(parent, original, name, condition):
new_sub_parser = argparse._SubParsersAction(
original.option_strings,
original._prog_prefix,
original._parser_class,
metavar=original.metavar
)
new_sub_parser._name_parser_map = original._name_parser_map
new_sub_parser._choices_actions = [
a for a in original._choices_actions if condition(original._name_parser_map[a.dest])
]
group = argparse._ArgumentGroup(parent, name)
group._group_actions = [new_sub_parser]
return group
class ArgumentParser(argparse.ArgumentParser):
def __init__(self, *args, group_name=None, **kwargs):
super().__init__(*args, formatter_class=HelpFormatter, add_help=False, **kwargs)
self.add_argument(
'--help', dest='help', action='store_true', default=False,
help='Show this help message and exit.'
)
self._optionals.title = 'Options'
if group_name is None:
self.epilog = (
"Run 'lbrynet COMMAND --help' for more information on a command or group."
)
else:
self.epilog = (
f"Run 'lbrynet {group_name} COMMAND --help' for more information on a command."
)
self.set_defaults(group=group_name, group_parser=self)
def format_help(self):
formatter = self._get_formatter()
formatter.add_usage(
self.usage, self._actions, self._mutually_exclusive_groups
)
formatter.add_text(self.description)
# positionals, optionals and user-defined groups
for action_group in self._granular_action_groups:
formatter.start_section(action_group.title)
formatter.add_text(action_group.description)
formatter.add_arguments(action_group._group_actions)
formatter.end_section()
formatter.add_text(self.epilog)
return formatter.format_help()
@property
def _granular_action_groups(self):
if self.prog != 'lbrynet':
yield from self._action_groups
return
yield self._optionals
action: argparse._SubParsersAction = self._positionals._group_actions[0]
yield split_subparser_argument(
self, action, "Grouped Commands", lambda parser: 'group' in parser._defaults
)
yield split_subparser_argument(
self, action, "Commands", lambda parser: 'group' not in parser._defaults
)
def error(self, message):
self.print_help(argparse._sys.stderr)
self.exit(2, f"\n{message}\n")
class HelpFormatter(argparse.HelpFormatter):
def add_usage(self, usage, actions, groups, prefix='Usage: '):
super().add_usage(
usage, [a for a in actions if a.option_strings != ['--help']], groups, prefix
)
def add_command_parser(parent, method_name, command):
short = command['desc']['text'][0] if command['desc'] else ''
subcommand = parent.add_parser(command['name'], help=short)
subcommand.set_defaults(api_method_name=method_name, command=command['name'], doc=command['help'])
def get_argument_parser():
root = ArgumentParser(
'lbrynet', description='An interface to the LBRY Network.', allow_abbrev=False,
)
root.add_argument(
'-v', '--version', dest='cli_version', action="store_true",
help='Show lbrynet CLI version and exit.'
)
root.set_defaults(group=None, command=None)
CLIConfig.contribute_to_argparse(root)
sub = root.add_subparsers(metavar='COMMAND')
start = sub.add_parser(
'start',
usage='lbrynet start [--config FILE] [--data-dir DIR] [--wallet-dir DIR] [--download-dir DIR] ...',
help='Start LBRY Network interface.'
)
start.add_argument(
"service", choices=[LightClient.name, FullNode.name], default=LightClient.name, nargs="?"
)
start.add_argument(
'--quiet', dest='quiet', action="store_true",
help='Disable all console output.'
)
start.add_argument(
'--no-logging', dest='no_logging', action="store_true",
help='Disable all logging of any kind.'
)
start.add_argument(
'--verbose', nargs="*",
help=('Enable debug output for lbry logger and event loop. Optionally specify loggers for which debug output '
'should selectively be applied.')
)
start.add_argument(
'--initial-headers', dest='initial_headers',
help='Specify path to initial blockchain headers, faster than downloading them on first run.'
)
install = sub.add_parser("install", help="Install lbrynet with various system services.")
install.add_argument("system", choices=["systemd"])
install.add_argument(
"--global", dest="install_global", action="store_true",
help="Install system wide (requires running as root), default is for current user only."
)
Config.contribute_to_argparse(start)
start.set_defaults(command='start', start_parser=start, doc=start.format_help())
install.set_defaults(command='install', install_parser=install, doc=install.format_help())
groups = {}
for group_name in sorted(interface['groups']):
group_parser = sub.add_parser(group_name, group_name=group_name, help=interface['groups'][group_name])
groups[group_name] = group_parser.add_subparsers(metavar='COMMAND')
nicer_order = ['stop', 'get', 'publish', 'resolve']
for command_name in sorted(interface['commands']):
if command_name not in nicer_order:
nicer_order.append(command_name)
for command_name in nicer_order:
command = interface['commands'][command_name]
if command.get('group') is None:
add_command_parser(sub, command_name, command)
else:
add_command_parser(groups[command['group']], command_name, command)
return root
def ensure_directory_exists(path: str):
if not os.path.isdir(path):
pathlib.Path(path).mkdir(parents=True, exist_ok=True)
async def execute_command(conf, method, params):
client = Client(f"http://{conf.api}/ws")
await client.connect()
responses = await client.send(method, **params)
result = await responses.first
await client.disconnect()
print(result)
return result
def normalize_value(x, key=None):
if not isinstance(x, str):
return x
if key in ('uri', 'channel_name', 'name', 'file_name', 'claim_name', 'download_directory'):
return x
if x.lower() == 'true':
return True
if x.lower() == 'false':
return False
if x.isdigit():
return int(x)
return x
def remove_brackets(key):
if key.startswith("<") and key.endswith(">"):
return str(key[1:-1])
return key
def set_kwargs(parsed_args):
kwargs = {}
for key, arg in parsed_args.items():
if arg is None:
continue
k = None
if key.startswith("--") and remove_brackets(key[2:]) not in kwargs:
k = remove_brackets(key[2:])
elif remove_brackets(key) not in kwargs:
k = remove_brackets(key)
kwargs[k] = normalize_value(arg, k)
return kwargs
def install_systemd_service():
systemd_service = textwrap.dedent(f"""\
[Unit]
Description=LBRYnet
[Service]
Type=simple
ExecStart={sys.argv[0]} start --full-node
""")
subprocess.run(
["systemctl", "edit", "--user", "--full", "--force", "lbrynet.service"],
input=systemd_service, text=True, check=True,
env=dict(os.environ, SYSTEMD_EDITOR="cp /dev/stdin"),
)
def main(argv=None):
argv = argv or sys.argv[1:]
parser = get_argument_parser()
args, command_args = parser.parse_known_args(argv)
conf = Config()
conf.set_arguments(args)
conf.set_environment()
conf.set_default_paths()
conf.set_persisted()
for directory in (conf.data_dir, conf.download_dir, conf.wallet_dir):
ensure_directory_exists(directory)
if args.cli_version:
print(f"lbrynet {__version__}")
elif args.command == 'start':
if args.help:
args.start_parser.print_help()
elif args.service == FullNode.name:
return Daemon.from_config(FullNode, conf).run()
else:
print(f'Only `start {FullNode.name}` is currently supported.')
elif args.command == 'install':
if args.help:
args.install_parser.print_help()
elif args.system == 'systemd':
install_systemd_service()
elif args.command is not None:
doc = args.doc
api_method_name = args.api_method_name
if args.help:
print(doc)
else:
parsed = docopt(
# TODO: ugly hack because docopt doesn't support commands with spaces in them
doc.replace(api_method_name.replace('_', ' '), api_method_name, 1),
command_args
)
params = set_kwargs(parsed)
asyncio.get_event_loop().run_until_complete(execute_command(conf, api_method_name, params))
elif args.group is not None:
args.group_parser.print_help()
else:
parser.print_help()
return 0
if __name__ == "__main__":
sys.exit(main())
+759
View File
@@ -0,0 +1,759 @@
import os
import sys
import typing
import logging
from argparse import ArgumentParser
from contextlib import contextmanager
from typing import Tuple
import yaml
from lbry.utils.dirs import user_data_dir, user_download_dir
from lbry.error import InvalidCurrencyError
from lbry.dht import constants
from lbry.wallet.coinselection import COIN_SELECTION_STRATEGIES
from lbry.event import EventRegistry
log = logging.getLogger(__name__)
NOT_SET = type('NOT_SET', (object,), {}) # pylint: disable=invalid-name
T = typing.TypeVar('T')
CURRENCIES = {
'BTC': {'type': 'crypto'},
'LBC': {'type': 'crypto'},
'USD': {'type': 'fiat'},
}
class Setting(typing.Generic[T]):
def __init__(self, doc: str, default: typing.Optional[T] = None,
previous_names: typing.Optional[typing.List[str]] = None,
metavar: typing.Optional[str] = None):
self.doc = doc
self.default = default
self.previous_names = previous_names or []
self.metavar = metavar
def __set_name__(self, owner, name):
self.name = name # pylint: disable=attribute-defined-outside-init
@property
def cli_name(self):
return f"--{self.name.replace('_', '-')}"
@property
def no_cli_name(self):
return f"--no-{self.name.replace('_', '-')}"
def __get__(self, obj: typing.Optional['BaseConfig'], owner) -> T:
if obj is None:
return self
for location in obj.search_order:
if self.name in location:
return location[self.name]
return self.default
def __set__(self, obj: 'BaseConfig', val: typing.Union[T, NOT_SET]):
if val == NOT_SET:
for location in obj.modify_order:
if self.name in location:
del location[self.name]
else:
self.validate(val)
for location in obj.modify_order:
location[self.name] = val
def validate(self, value):
raise NotImplementedError()
def deserialize(self, value): # pylint: disable=no-self-use
return value
def serialize(self, value): # pylint: disable=no-self-use
return value
def contribute_to_argparse(self, parser: ArgumentParser):
parser.add_argument(
self.cli_name,
help=self.doc,
metavar=self.metavar,
default=NOT_SET
)
class String(Setting[str]):
def validate(self, value):
assert isinstance(value, str), \
f"Setting '{self.name}' must be a string."
# TODO: removes this after pylint starts to understand generics
def __get__(self, obj: typing.Optional['BaseConfig'], owner) -> str: # pylint: disable=useless-super-delegation
return super().__get__(obj, owner)
class Integer(Setting[int]):
def validate(self, value):
assert isinstance(value, int), \
f"Setting '{self.name}' must be an integer."
def deserialize(self, value):
return int(value)
class Float(Setting[float]):
def validate(self, value):
assert isinstance(value, float), \
f"Setting '{self.name}' must be a decimal."
def deserialize(self, value):
return float(value)
class Toggle(Setting[bool]):
def validate(self, value):
assert isinstance(value, bool), \
f"Setting '{self.name}' must be a true/false value."
def contribute_to_argparse(self, parser: ArgumentParser):
parser.add_argument(
self.cli_name,
help=self.doc,
action="store_true",
default=NOT_SET
)
parser.add_argument(
self.no_cli_name,
help=f"Opposite of {self.cli_name}",
dest=self.name,
action="store_false",
default=NOT_SET
)
class Path(String):
def __init__(self, doc: str, *args, default: str = '', **kwargs):
super().__init__(doc, default, *args, **kwargs)
def __get__(self, obj, owner) -> str:
value = super().__get__(obj, owner)
if isinstance(value, str):
return os.path.expanduser(os.path.expandvars(value))
return value
class MaxKeyFee(Setting[dict]):
def validate(self, value):
if value is not None:
assert isinstance(value, dict) and set(value) == {'currency', 'amount'}, \
f"Setting '{self.name}' must be a dict like \"{{'amount': 50.0, 'currency': 'USD'}}\"."
if value["currency"] not in CURRENCIES:
raise InvalidCurrencyError(value["currency"])
@staticmethod
def _parse_list(l):
if l == ['null']:
return None
assert len(l) == 2, (
'Max key fee is made up of either two values: '
'"AMOUNT CURRENCY", or "null" (to set no limit)'
)
try:
amount = float(l[0])
except ValueError:
raise AssertionError('First value in max key fee is a decimal: "AMOUNT CURRENCY"')
currency = str(l[1]).upper()
if currency not in CURRENCIES:
raise InvalidCurrencyError(currency)
return {'amount': amount, 'currency': currency}
def deserialize(self, value):
if value is None:
return
if isinstance(value, dict):
return {
'currency': value['currency'],
'amount': float(value['amount']),
}
if isinstance(value, str):
value = value.split()
if isinstance(value, list):
return self._parse_list(value)
raise AssertionError('Invalid max key fee.')
def contribute_to_argparse(self, parser: ArgumentParser):
parser.add_argument(
self.cli_name,
help=self.doc,
nargs='+',
metavar=('AMOUNT', 'CURRENCY'),
default=NOT_SET
)
parser.add_argument(
self.no_cli_name,
help="Disable maximum key fee check.",
dest=self.name,
const=None,
action="store_const",
default=NOT_SET
)
class StringChoice(String):
def __init__(self, doc: str, valid_values: typing.List[str], default: str, *args, **kwargs):
super().__init__(doc, default, *args, **kwargs)
if not valid_values:
raise ValueError("No valid values provided")
if default not in valid_values:
raise ValueError(f"Default value must be one of: {', '.join(valid_values)}")
self.valid_values = valid_values
def validate(self, value):
super().validate(value)
if value not in self.valid_values:
raise ValueError(f"Setting '{self.name}' value must be one of: {', '.join(self.valid_values)}")
class ListSetting(Setting[list]):
def validate(self, value):
assert isinstance(value, (tuple, list)), \
f"Setting '{self.name}' must be a tuple or list."
def contribute_to_argparse(self, parser: ArgumentParser):
parser.add_argument(
self.cli_name,
help=self.doc,
action='append'
)
class Servers(ListSetting):
def validate(self, value):
assert isinstance(value, (tuple, list)), \
f"Setting '{self.name}' must be a tuple or list of servers."
for idx, server in enumerate(value):
assert isinstance(server, (tuple, list)) and len(server) == 2, \
f"Server defined '{server}' at index {idx} in setting " \
f"'{self.name}' must be a tuple or list of two items."
assert isinstance(server[0], str), \
f"Server defined '{server}' at index {idx} in setting " \
f"'{self.name}' must be have hostname as string in first position."
assert isinstance(server[1], int), \
f"Server defined '{server}' at index {idx} in setting " \
f"'{self.name}' must be have port as int in second position."
def deserialize(self, value):
servers = []
if isinstance(value, list):
for server in value:
if isinstance(server, str) and server.count(':') == 1:
host, port = server.split(':')
try:
servers.append((host, int(port)))
except ValueError:
pass
return servers
def serialize(self, value):
if value:
return [f"{host}:{port}" for host, port in value]
return value
class Strings(ListSetting):
def validate(self, value):
assert isinstance(value, (tuple, list)), \
f"Setting '{self.name}' must be a tuple or list of strings."
for idx, string in enumerate(value):
assert isinstance(string, str), \
f"Value of '{string}' at index {idx} in setting " \
f"'{self.name}' must be a string."
class EnvironmentAccess:
PREFIX = 'LBRY_'
def __init__(self, environ: dict):
self.environ = environ
def __contains__(self, item: str):
return f'{self.PREFIX}{item.upper()}' in self.environ
def __getitem__(self, item: str):
return self.environ[f'{self.PREFIX}{item.upper()}']
class ArgumentAccess:
def __init__(self, config: 'BaseConfig', args: dict):
self.configuration = config
self.args = {}
if args:
self.load(args)
def load(self, args):
for setting in self.configuration.get_settings():
value = getattr(args, setting.name, NOT_SET)
if value != NOT_SET and not (isinstance(setting, ListSetting) and value is None):
self.args[setting.name] = setting.deserialize(value)
def __contains__(self, item: str):
return item in self.args
def __getitem__(self, item: str):
return self.args[item]
class ConfigFileAccess:
def __init__(self, config: 'BaseConfig', path: str):
self.configuration = config
self.path = path
self.data = {}
if self.exists:
self.load()
@property
def exists(self):
return self.path and os.path.exists(self.path)
def load(self):
cls = type(self.configuration)
with open(self.path, 'r') as config_file:
raw = config_file.read()
serialized = yaml.full_load(raw) or {}
for key, value in serialized.items():
attr = getattr(cls, key, None)
if attr is None:
for setting in self.configuration.settings:
if key in setting.previous_names:
attr = setting
break
if attr is not None:
self.data[key] = attr.deserialize(value)
def save(self):
cls = type(self.configuration)
serialized = {}
for key, value in self.data.items():
attr = getattr(cls, key)
serialized[key] = attr.serialize(value)
with open(self.path, 'w') as config_file:
config_file.write(yaml.safe_dump(serialized, default_flow_style=False))
def upgrade(self) -> bool:
upgraded = False
for key in list(self.data):
for setting in self.configuration.settings:
if key in setting.previous_names:
self.data[setting.name] = self.data[key]
del self.data[key]
upgraded = True
break
return upgraded
def __contains__(self, item: str):
return item in self.data
def __getitem__(self, item: str):
return self.data[item]
def __setitem__(self, key, value):
self.data[key] = value
def __delitem__(self, key):
del self.data[key]
TBC = typing.TypeVar('TBC', bound='BaseConfig')
class BaseConfig:
config = Path("Path to configuration file.", metavar='FILE')
def __init__(self, **kwargs):
self.runtime = {} # set internally or by various API calls
self.arguments = {} # from command line arguments
self.environment = {} # from environment variables
self.persisted = {} # from config file
self._updating_config = False
self.events = EventRegistry()
self.set(**kwargs)
def set(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
return self
@contextmanager
def update_config(self):
self._updating_config = True
yield self
self._updating_config = False
if isinstance(self.persisted, ConfigFileAccess):
self.persisted.save()
@property
def modify_order(self):
locations = [self.runtime]
if self._updating_config:
locations.append(self.persisted)
return locations
@property
def search_order(self):
return [
self.runtime,
self.arguments,
self.environment,
self.persisted
]
@classmethod
def get_settings(cls):
for attr in dir(cls):
setting = getattr(cls, attr)
if isinstance(setting, Setting):
yield setting
@property
def settings(self):
return self.get_settings()
@property
def settings_dict(self):
return {
setting.name: getattr(self, setting.name) for setting in self.settings
}
@classmethod
def create_from_arguments(cls, args) -> TBC:
conf = cls()
conf.set_arguments(args)
conf.set_environment()
conf.set_persisted()
return conf
@classmethod
def contribute_to_argparse(cls, parser: ArgumentParser):
for setting in cls.get_settings():
setting.contribute_to_argparse(parser)
def set_arguments(self, args):
self.arguments = ArgumentAccess(self, args)
def set_environment(self, environ=None):
self.environment = EnvironmentAccess(environ or os.environ)
def set_persisted(self, config_file_path=None):
if config_file_path is None:
config_file_path = self.config
if not config_file_path:
return
ext = os.path.splitext(config_file_path)[1]
assert ext in ('.yml', '.yaml'),\
f"File extension '{ext}' is not supported, " \
f"configuration file must be in YAML (.yaml)."
self.persisted = ConfigFileAccess(self, config_file_path)
if self.persisted.upgrade():
self.persisted.save()
class TranscodeConfig(BaseConfig):
ffmpeg_path = String('A list of places to check for ffmpeg and ffprobe. '
f'$data_dir/ffmpeg/bin and $PATH are checked afterward. Separator: {os.pathsep}',
'', previous_names=['ffmpeg_folder'])
video_encoder = String('FFmpeg codec and parameters for the video encoding. '
'Example: libaom-av1 -crf 25 -b:v 0 -strict experimental',
'libx264 -crf 21 -preset faster -pix_fmt yuv420p')
video_bitrate_maximum = Integer('Maximum bits per second allowed for video streams (0 to disable).', 8400000)
video_scaler = String('FFmpeg scaling parameters for reducing bitrate. '
'Example: -vf "scale=-2:720,fps=24" -maxrate 5M -bufsize 3M',
r'-vf "scale=if(gte(iw\,ih)\,min(1920\,iw)\,-2):if(lt(iw\,ih)\,min(1920\,ih)\,-2)" '
r'-maxrate 8400K -bufsize 5000K')
audio_encoder = String('FFmpeg codec and parameters for the audio encoding. '
'Example: libopus -b:a 128k',
'aac -b:a 160k')
volume_filter = String('FFmpeg filter for audio normalization.', '-af loudnorm')
volume_analysis_time = Integer('Maximum seconds into the file that we examine audio volume (0 to disable).', 240)
class CLIConfig(TranscodeConfig):
api = String('Host name and port for lbrynet daemon API.', 'localhost:5279', metavar='HOST:PORT')
@property
def api_connection_url(self) -> str:
return f"http://{self.api}/lbryapi"
@property
def api_host(self):
return self.api.split(':')[0]
@property
def api_port(self):
return int(self.api.split(':')[1])
class Config(CLIConfig):
db_url = String("Database connection URL, uses a local file based SQLite by default.")
workers = Integer(
"Multiprocessing, specify number of worker processes lbrynet can start (including main process)."
" (-1: threads only, 0: equal to number of CPUs, >1: specific number of processes)", -1
)
console = StringChoice(
"Basic text console output or advanced colored output with progress bars.",
["basic", "advanced", "none"], "advanced"
)
# directories
download_dir = Path("Directory to store downloaded files.", metavar='DIR')
data_dir = Path("Main directory containing blobs, wallets and blockchain data.", metavar='DIR')
blob_dir = Path("Directory to store blobs (default: 'data_dir'/blobs).", metavar='DIR')
wallet_dir = Path("Directory to store wallets (default: 'data_dir'/wallets).", metavar='DIR')
wallet_storage = StringChoice("Wallet storage mode.", ["file", "database"], "file")
wallets = Strings(
"Wallet files in 'wallet_dir' to load at startup.", ['default_wallet']
)
create_default_wallet = Toggle(
"Create an initial wallet if it does not exist on startup.", True
)
create_default_account = Toggle(
"Create an initial account if it does not exist in the default wallet.", True
)
# network
use_upnp = Toggle(
"Use UPnP to setup temporary port redirects for the DHT and the hosting of blobs. If you manually forward"
"ports or have firewall rules you likely want to disable this.", True
)
udp_port = Integer("UDP port for communicating on the LBRY DHT", 4444, previous_names=['dht_node_port'])
tcp_port = Integer("TCP port to listen for incoming blob requests", 3333, previous_names=['peer_port'])
prometheus_port = Integer("Port to expose prometheus metrics (off by default)", 0)
network_interface = String("Interface to use for the DHT and blob exchange", '0.0.0.0')
# routing table
split_buckets_under_index = Integer(
"Routing table bucket index below which we always split the bucket if given a new key to add to it and "
"the bucket is full. As this value is raised the depth of the routing table (and number of peers in it) "
"will increase. This setting is used by seed nodes, you probably don't want to change it during normal "
"use.", 1
)
# protocol timeouts
download_timeout = Float("Cumulative timeout for a stream to begin downloading before giving up", 30.0)
blob_download_timeout = Float("Timeout to download a blob from a peer", 30.0)
peer_connect_timeout = Float("Timeout to establish a TCP connection to a peer", 3.0)
node_rpc_timeout = Float("Timeout when making a DHT request", constants.RPC_TIMEOUT)
# blob announcement and download
save_blobs = Toggle("Save encrypted blob files for hosting, otherwise download blobs to memory only.", True)
blob_lru_cache_size = Integer(
"LRU cache size for decrypted downloaded blobs used to minimize re-downloading the same blobs when "
"replying to a range request. Set to 0 to disable.", 32
)
announce_head_and_sd_only = Toggle(
"Announce only the descriptor and first (rather than all) data blob for a stream to the DHT", True,
previous_names=['announce_head_blobs_only']
)
concurrent_blob_announcers = Integer(
"Number of blobs to iteratively announce at once, set to 0 to disable", 10,
previous_names=['concurrent_announcers']
)
max_connections_per_download = Integer(
"Maximum number of peers to connect to while downloading a blob", 4,
previous_names=['max_connections_per_stream']
)
fixed_peer_delay = Float(
"Amount of seconds before adding the reflector servers as potential peers to download from in case dht"
"peers are not found or are slow", 2.0
)
max_key_fee = MaxKeyFee(
"Don't download streams with fees exceeding this amount. When set to "
"null, the amount is unbounded.", {'currency': 'USD', 'amount': 50.0}
)
max_wallet_server_fee = String("Maximum daily LBC amount allowed as payment for wallet servers.", "0.0")
# reflector settings
reflect_streams = Toggle(
"Upload completed streams (published and downloaded) reflector in order to re-host them", True,
previous_names=['reflect_uploads']
)
concurrent_reflector_uploads = Integer(
"Maximum number of streams to upload to a reflector server at a time", 10
)
# servers
reflector_servers = Servers("Reflector re-hosting servers", [
('reflector.lbry.com', 5566)
])
known_full_nodes = Servers("Full blockchain nodes", [
('spv11.lbry.com', 50001),
('spv12.lbry.com', 50001),
('spv13.lbry.com', 50001),
('spv14.lbry.com', 50001),
('spv15.lbry.com', 50001),
('spv16.lbry.com', 50001),
('spv17.lbry.com', 50001),
('spv18.lbry.com', 50001),
('spv19.lbry.com', 50001),
])
known_dht_nodes = Servers("Known nodes for bootstrapping connection to the DHT", [
('lbrynet1.lbry.com', 4444), # US EAST
('lbrynet2.lbry.com', 4444), # US WEST
('lbrynet3.lbry.com', 4444), # EU
('lbrynet4.lbry.com', 4444) # ASIA
])
comment_server = String("Comment server API URL", "https://comments.lbry.com/api")
# blockchain
blockchain = StringChoice("Blockchain network type.", ["mainnet", "regtest", "testnet"], "mainnet")
lbrycrd_rpc_user = String("Username for connecting to lbrycrd.", "rpcuser")
lbrycrd_rpc_pass = String("Password for connecting to lbrycrd.", "rpcpassword")
lbrycrd_rpc_host = String("Hostname for connecting to lbrycrd.", "localhost")
lbrycrd_rpc_port = Integer("Port for connecting to lbrycrd.", 9245)
lbrycrd_peer_port = Integer("Peer port for lbrycrd.", 9246)
lbrycrd_zmq = String("ZMQ events address.")
lbrycrd_dir = Path("Directory containing lbrycrd data.", metavar='DIR')
search_censor_channel_ids = Strings("List of channel ids for filtering out search results.", [])
resolve_censor_channel_ids = Strings("List of channel ids for filtering out resolve results.", [])
# daemon
save_files = Toggle("Save downloaded files when calling `get` by default", True)
components_to_skip = Strings("components which will be skipped during start-up of daemon", [])
share_usage_data = Toggle(
"Whether to share usage stats and diagnostic info with LBRY.", False,
previous_names=['upload_log', 'upload_log', 'share_debug_info']
)
track_bandwidth = Toggle("Track bandwidth usage", True)
# media server
streaming_server = String('Host name and port to serve streaming media over range requests',
'localhost:5280', metavar='HOST:PORT')
streaming_get = Toggle("Enable the /get endpoint for the streaming media server. "
"Disable to prevent new streams from being added.", True)
coin_selection_strategy = StringChoice(
"Strategy to use when selecting UTXOs for a transaction",
COIN_SELECTION_STRATEGIES, "standard")
save_resolved_claims = Toggle(
"Save content claims to the database when they are resolved to keep file_list up to date, "
"only disable this if file_x commands are not needed", True
)
@property
def streaming_host(self):
return self.streaming_server.split(':')[0]
@property
def streaming_port(self):
return int(self.streaming_server.split(':')[1])
@classmethod
def with_null_dir(cls):
return cls.with_same_dir('/dev/null')
@classmethod
def with_same_dir(cls, same_dir):
return cls(
data_dir=same_dir,
download_dir=same_dir,
wallet_dir=same_dir,
lbrycrd_dir=same_dir,
)
def set_default_paths(self):
if 'darwin' in sys.platform.lower():
get_directories = get_darwin_directories
elif 'win' in sys.platform.lower():
get_directories = get_windows_directories
elif 'linux' in sys.platform.lower():
get_directories = get_linux_directories
else:
return
cls = type(self)
cls.data_dir.default, cls.wallet_dir.default,\
cls.blob_dir.default, cls.download_dir.default = get_directories()
old_settings_file = os.path.join(self.data_dir, 'daemon_settings.yml')
if os.path.exists(old_settings_file):
cls.config.default = old_settings_file
else:
cls.config.default = os.path.join(self.data_dir, 'settings.yml')
if self.data_dir != cls.data_dir.default:
cls.blob_dir.default = os.path.join(self.data_dir, 'blobs')
cls.wallet_dir.default = os.path.join(self.data_dir, 'wallets')
@property
def log_file_path(self):
return os.path.join(self.data_dir, 'daemon.log')
@property
def db_url_or_default(self):
if self.db_url:
return self.db_url
return 'sqlite:///'+os.path.join(self.data_dir, f'{self.blockchain}.db')
def get_windows_directories() -> Tuple[str, str, str, str]:
# very old
data_dir = user_data_dir('lbrynet', roaming=True)
blob_dir = os.path.join(data_dir, 'blobfiles')
wallet_dir = os.path.join(user_data_dir('lbryum', roaming=True), 'wallets')
if os.path.isdir(blob_dir) or os.path.isdir(wallet_dir):
return data_dir, wallet_dir, blob_dir, user_download_dir()
# old
data_dir = user_data_dir('lbrynet', 'lbry')
blob_dir = os.path.join(data_dir, 'blobfiles')
wallet_dir = os.path.join(user_data_dir('lbryum', 'lbry'), 'wallets')
if os.path.isdir(blob_dir) and os.path.isdir(wallet_dir):
return data_dir, wallet_dir, blob_dir, user_download_dir()
# new
return get_universal_directories()
def get_darwin_directories() -> Tuple[str, str, str, str]:
data_dir = user_data_dir('LBRY')
blob_dir = os.path.join(data_dir, 'blobfiles')
wallet_dir = os.path.expanduser('~/.lbryum/wallets')
if os.path.isdir(blob_dir) or os.path.isdir(wallet_dir):
return data_dir, wallet_dir, blob_dir, user_download_dir()
return get_universal_directories()
def get_linux_directories() -> Tuple[str, str, str, str]:
# very old
data_dir = os.path.expanduser('~/.lbrynet')
blob_dir = os.path.join(data_dir, 'blobfiles')
wallet_dir = os.path.join(os.path.expanduser('~/.lbryum'), 'wallets')
if os.path.isdir(blob_dir) or os.path.isdir(wallet_dir):
return data_dir, wallet_dir, blob_dir, user_download_dir()
# old
data_dir = user_data_dir('lbry/lbrynet')
blob_dir = os.path.join(data_dir, 'blobfiles')
wallet_dir = user_data_dir('lbry/lbryum/wallets')
if os.path.isdir(blob_dir) or os.path.isdir(wallet_dir):
return data_dir, wallet_dir, blob_dir, user_download_dir()
# new
return get_universal_directories()
def get_universal_directories() -> Tuple[str, str, str, str]:
lbrynet_dir = user_data_dir('lbrynet', 'LBRY')
return (
lbrynet_dir,
os.path.join(lbrynet_dir, 'wallets'),
os.path.join(lbrynet_dir, 'blobs'),
user_download_dir()
)
+105
View File
@@ -0,0 +1,105 @@
import time
import asyncio
import typing
import collections
import logging
log = logging.getLogger(__name__)
CONNECTED_EVENT = "connected"
DISCONNECTED_EVENT = "disconnected"
TRANSFERRED_EVENT = "transferred"
class ConnectionManager:
def __init__(self, loop: asyncio.AbstractEventLoop):
self.loop = loop
self.incoming_connected: typing.Set[str] = set()
self.incoming: typing.DefaultDict[str, int] = collections.defaultdict(int)
self.outgoing_connected: typing.Set[str] = set()
self.outgoing: typing.DefaultDict[str, int] = collections.defaultdict(int)
self._max_incoming_mbs = 0.0
self._max_outgoing_mbs = 0.0
self._status = {}
self._running = False
self._task: typing.Optional[asyncio.Task] = None
@property
def status(self):
return self._status
def sent_data(self, host_and_port: str, size: int):
if self._running:
self.outgoing[host_and_port] += size
def received_data(self, host_and_port: str, size: int):
if self._running:
self.incoming[host_and_port] += size
def connection_made(self, host_and_port: str):
if self._running:
self.outgoing_connected.add(host_and_port)
def connection_received(self, host_and_port: str):
# self.incoming_connected.add(host_and_port)
pass
def outgoing_connection_lost(self, host_and_port: str):
if self._running and host_and_port in self.outgoing_connected:
self.outgoing_connected.remove(host_and_port)
def incoming_connection_lost(self, host_and_port: str):
if self._running and host_and_port in self.incoming_connected:
self.incoming_connected.remove(host_and_port)
async def _update(self):
self._status = {
'incoming_bps': {},
'outgoing_bps': {},
'total_incoming_mbs': 0.0,
'total_outgoing_mbs': 0.0,
'total_sent': 0,
'total_received': 0,
'max_incoming_mbs': 0.0,
'max_outgoing_mbs': 0.0
}
while True:
last = time.perf_counter()
await asyncio.sleep(0.1, loop=self.loop)
self._status['incoming_bps'].clear()
self._status['outgoing_bps'].clear()
now = time.perf_counter()
while self.outgoing:
k, sent = self.outgoing.popitem()
self._status['total_sent'] += sent
self._status['outgoing_bps'][k] = sent / (now - last)
while self.incoming:
k, received = self.incoming.popitem()
self._status['total_received'] += received
self._status['incoming_bps'][k] = received / (now - last)
self._status['total_outgoing_mbs'] = int(sum(list(self._status['outgoing_bps'].values())
)) / 1000000.0
self._status['total_incoming_mbs'] = int(sum(list(self._status['incoming_bps'].values())
)) / 1000000.0
self._max_incoming_mbs = max(self._max_incoming_mbs, self._status['total_incoming_mbs'])
self._max_outgoing_mbs = max(self._max_outgoing_mbs, self._status['total_outgoing_mbs'])
self._status['max_incoming_mbs'] = self._max_incoming_mbs
self._status['max_outgoing_mbs'] = self._max_outgoing_mbs
def stop(self):
if self._task:
self._task.cancel()
self._task = None
self.outgoing.clear()
self.outgoing_connected.clear()
self.incoming.clear()
self.incoming_connected.clear()
self._status.clear()
self._running = False
def start(self):
self.stop()
self._running = True
self._task = self.loop.create_task(self._update())
+496
View File
@@ -0,0 +1,496 @@
import os
import sys
import time
import itertools
import logging
from typing import Dict, Any, Type
from tempfile import TemporaryFile
from tqdm.std import tqdm, Bar
from tqdm.utils import FormatReplace, _unicode, disp_len, disp_trim, _is_ascii
from lbry import __version__
from lbry.service.base import Service
from lbry.service.full_node import FullNode
from lbry.service.light_client import LightClient
log = logging.getLogger(__name__)
class RedirectOutput:
silence_lines = [
b'libprotobuf ERROR google/protobuf/wire_format_lite.cc:626',
]
def __init__(self, stream_type: str):
assert stream_type in ('stderr', 'stdout')
self.stream_type = stream_type
self.stream_no = getattr(sys, stream_type).fileno()
self.last_flush = time.time()
self.last_read = 0
self.backup = None
self.file = None
def __enter__(self):
self.backup = os.dup(self.stream_no)
setattr(sys, self.stream_type, os.fdopen(self.backup, 'w'))
self.file = TemporaryFile()
self.backup = os.dup2(self.file.fileno(), self.stream_no)
def __exit__(self, exc_type, exc_val, exc_tb):
self.file.close()
os.dup2(self.backup, self.stream_no)
os.close(self.backup)
setattr(sys, self.stream_type, os.fdopen(self.stream_no, 'w'))
def capture(self):
self.__enter__()
def release(self):
self.__exit__(None, None, None)
def flush(self, writer, force=False):
if not force and (time.time() - self.last_flush) < 5:
return
self.file.seek(self.last_read)
for line in self.file.readlines():
silence = False
for bad_line in self.silence_lines:
if bad_line in line:
silence = True
break
if not silence:
writer(line.decode().rstrip())
self.last_read = self.file.tell()
self.last_flush = time.time()
class Console:
def __init__(self, service: Service):
self.service = service
def starting(self):
pass
def stopping(self):
pass
class Basic(Console):
def __init__(self, service: Service):
super().__init__(service)
self.service.sync.on_progress.listen(self.on_sync_progress)
self.tasks = {}
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)-8s %(name)s:%(lineno)d: %(message)s")
def starting(self):
conf = self.service.conf
s = [f'LBRY v{__version__}']
if isinstance(self.service, FullNode):
s.append('Full Node')
elif isinstance(self.service, LightClient):
s.append('Light Client')
if conf.workers == -1:
s.append('Threads Only')
else:
workers = os.cpu_count() if conf.workers == 0 else conf.workers
s.append(f'{workers} Worker' if workers == 1 else f'{workers} Workers')
s.append(f'({os.cpu_count()} CPUs available)')
log.info(' '.join(s))
def stopping(self):
log.info('exiting')
@staticmethod
def maybe_log_progress(event, done, total, last):
if done == 0:
log.info("%s 0%%", event)
return 0
elif done == total:
log.info("%s 100%%", event)
return 1
else:
percent = done/total
if percent >= 0.25 > last:
log.info("%s 25%%", event)
return 0.25
elif percent >= 0.50 > last:
log.info("%s 50%%", event)
return 0.50
elif percent >= 0.75 > last:
log.info("%s 75%%", event)
return 0.75
return last
def on_sync_progress(self, event):
e, data = event["event"], event["data"]
name, current, total, last = e, data['done'][0], 0, 0
if not e.endswith("init") and not e.endswith("main") and not e.endswith("indexes"):
name = f"{e}#{data['id']}"
if "total" in data:
total, last = self.tasks[name] = (data["total"][0], last)
elif name in self.tasks:
total, last = self.tasks[name]
elif total == 0:
return
progress_status = (total, self.maybe_log_progress(name, current, total, last))
if progress_status[1] == 1:
del self.tasks[name]
else:
self.tasks[name] = progress_status
class Bar2(Bar):
def __init__(self, frac, default_len=10, charset=None):
super().__init__(frac[0], default_len, charset)
self.frac2 = frac[1]
def __format__(self, format_spec):
width = self.default_len
row1 = (1,)*int(self.frac2 * width * 2)
row2 = (2,)*int(self.frac * width * 2)
fill = []
for one, two, _ in itertools.zip_longest(row1, row2, range(width*2)):
fill.append((one or 0)+(two or 0))
bar = []
for i in range(0, width*2, 2):
if fill[i] == 1:
if fill[i+1] == 1:
bar.append('▀')
else:
bar.append('▘')
elif fill[i] == 2:
if fill[i+1] == 2:
bar.append('▄')
else:
bar.append('▖')
elif fill[i] == 3:
if fill[i+1] == 1:
bar.append('▛')
elif fill[i+1] == 2:
bar.append('▙')
elif fill[i+1] == 3:
bar.append('█')
else:
bar.append('▌')
else:
bar.append(' ')
return ''.join(bar)
class tqdm2(tqdm): # pylint: disable=invalid-name
def __init__(self, initial=(0, 0), unit=('it', 'it'), total=(None, None), **kwargs):
self.n2 = self.last_print_n2 = initial[1] # pylint: disable=invalid-name
self.unit2 = unit[1]
self.total2 = total[1]
super().__init__(initial=initial[0], unit=unit[0], total=total[0], **kwargs)
@property
def format_dict(self):
d = super().format_dict
d.update({
'n2': self.n2,
'unit2': self.unit2,
'total2': self.total2,
})
return d
def update(self, n=(1, 1)):
if self.disable:
return
last_last_print_t = self.last_print_t
self.n2 += n[1]
super().update(n[0])
if last_last_print_t != self.last_print_t:
self.last_print_n2 = self.n2
@staticmethod
def format_meter(
n, total, elapsed, ncols=None, prefix='', ascii=False, # pylint: disable=redefined-builtin
unit='it', unit_scale=False, rate=None, bar_format=None,
postfix=None, unit_divisor=1000, initial=0, **extra_kwargs
):
# sanity check: total
if total and n >= (total + 0.5): # allow float imprecision (#849)
total = None
# apply custom scale if necessary
if unit_scale and unit_scale not in (True, 1):
if total:
total *= unit_scale
n *= unit_scale
if rate:
rate *= unit_scale # by default rate = 1 / self.avg_time
unit_scale = False
elapsed_str = tqdm.format_interval(elapsed)
# if unspecified, attempt to use rate = average speed
# (we allow manual override since predicting time is an arcane art)
if rate is None and elapsed:
rate = n / elapsed
inv_rate = 1 / rate if rate else None
format_sizeof = tqdm.format_sizeof
rate_noinv_fmt = ((format_sizeof(rate) if unit_scale else
'{0:5.2f}'.format(rate))
if rate else '?') + unit + '/s'
rate_inv_fmt = ((format_sizeof(inv_rate) if unit_scale else
'{0:5.2f}'.format(inv_rate))
if inv_rate else '?') + 's/' + unit
rate_fmt = rate_inv_fmt if inv_rate and inv_rate > 1 else rate_noinv_fmt
if unit_scale:
n_fmt = format_sizeof(n, divisor=unit_divisor)
total_fmt = format_sizeof(total, divisor=unit_divisor) \
if total is not None else '?'
else:
n_fmt = str(n)
total_fmt = str(total) if total is not None else '?'
try:
postfix = ', ' + postfix if postfix else ''
except TypeError:
pass
remaining = (total - n) / rate if rate and total else 0
remaining_str = tqdm.format_interval(remaining) if rate else '?'
# format the stats displayed to the left and right sides of the bar
if prefix:
# old prefix setup work around
bool_prefix_colon_already = (prefix[-2:] == ": ")
l_bar = prefix if bool_prefix_colon_already else prefix + ": "
else:
l_bar = ''
r_bar = '| {0}/{1} [{2}<{3}, {4}{5}]'.format(
n_fmt, total_fmt, elapsed_str, remaining_str, rate_fmt, postfix)
# Custom bar formatting
# Populate a dict with all available progress indicators
format_dict = dict(
# slight extension of self.format_dict
n=n, n_fmt=n_fmt, total=total, total_fmt=total_fmt,
elapsed=elapsed_str, elapsed_s=elapsed,
ncols=ncols, desc=prefix or '', unit=unit,
rate=inv_rate if inv_rate and inv_rate > 1 else rate,
rate_fmt=rate_fmt, rate_noinv=rate,
rate_noinv_fmt=rate_noinv_fmt, rate_inv=inv_rate,
rate_inv_fmt=rate_inv_fmt,
postfix=postfix, unit_divisor=unit_divisor,
# plus more useful definitions
remaining=remaining_str, remaining_s=remaining,
l_bar=l_bar, r_bar=r_bar,
**extra_kwargs)
# total is known: we can predict some stats
if total:
n2, total2 = extra_kwargs['n2'], extra_kwargs['total2'] # pylint: disable=invalid-name
# fractional and percentage progress
frac = n / total
frac2 = n2 / total2
percentage = frac * 100
l_bar += '{0:3.0f}%|'.format(percentage)
if ncols == 0:
return l_bar[:-1] + r_bar[1:]
format_dict.update(l_bar=l_bar)
if bar_format:
format_dict.update(percentage=percentage)
# auto-remove colon for empty `desc`
if not prefix:
bar_format = bar_format.replace("{desc}: ", '')
else:
bar_format = "{l_bar}{bar}{r_bar}"
full_bar = FormatReplace()
try:
nobar = bar_format.format(bar=full_bar, **format_dict)
except UnicodeEncodeError:
bar_format = _unicode(bar_format)
nobar = bar_format.format(bar=full_bar, **format_dict)
if not full_bar.format_called:
# no {bar}, we can just format and return
return nobar
# Formatting progress bar space available for bar's display
full_bar = Bar2(
(frac, frac2),
max(1, ncols - disp_len(nobar))
if ncols else 10,
charset=Bar2.ASCII if ascii is True else ascii or Bar2.UTF)
if not _is_ascii(full_bar.charset) and _is_ascii(bar_format):
bar_format = _unicode(bar_format)
res = bar_format.format(bar=full_bar, **format_dict)
return disp_trim(res, ncols) if ncols else res
elif bar_format:
# user-specified bar_format but no total
l_bar += '|'
format_dict.update(l_bar=l_bar, percentage=0)
full_bar = FormatReplace()
nobar = bar_format.format(bar=full_bar, **format_dict)
if not full_bar.format_called:
return nobar
full_bar = Bar2(
(0, 0),
max(1, ncols - disp_len(nobar))
if ncols else 10,
charset=Bar2.BLANK)
res = bar_format.format(bar=full_bar, **format_dict)
return disp_trim(res, ncols) if ncols else res
else:
# no total: no progressbar, ETA, just progress stats
return ((prefix + ": ") if prefix else '') + \
'{0}{1} [{2}, {3}{4}]'.format(
n_fmt, unit, elapsed_str, rate_fmt, postfix)
class Advanced(Basic):
FORMAT = '{l_bar}{bar}| {n_fmt:>8}/{total_fmt:>8} [{elapsed:>7}<{remaining:>8}, {rate_fmt:>17}]'
def __init__(self, service: Service):
super().__init__(service)
self.bars: Dict[Any, tqdm] = {}
self.stderr = RedirectOutput('stderr')
def starting(self):
self.stderr.capture()
super().starting()
def stopping(self):
for bar in self.bars.values():
bar.close()
super().stopping()
#self.stderr.flush(self.bars['read'].write, True)
#self.stderr.release()
def get_or_create_bar(self, name, desc, units, totals, leave=False, bar_format=None, postfix=None, position=None):
bar = self.bars.get(name)
if bar is None:
if len(units) == 2:
bar = self.bars[name] = tqdm2(
desc=desc, unit=units, total=totals,
bar_format=bar_format or self.FORMAT, leave=leave,
postfix=postfix, position=position
)
else:
bar = self.bars[name] = tqdm(
desc=desc, unit=units[0], total=totals[0],
bar_format=bar_format or self.FORMAT, leave=leave,
postfix=postfix, position=position
)
return bar
def sync_init(self, name, d):
bar_name = f"{name}#{d['id']}"
bar = self.bars.get(bar_name)
if bar is None:
label = d.get('label', name[-11:])
self.get_or_create_bar(bar_name, label, d['units'], d['total'], True)
else:
if d['done'][0] != -1:
bar.update(d['done'][0] - bar.n)
if d['done'][0] == -1 or d['done'][0] == bar.total:
bar.close()
self.bars.pop(bar_name)
def sync_main(self, name, d):
bar = self.bars.get(name)
if bar is None:
label = d.get('label', name[-11:])
self.get_or_create_bar(name, label, d['units'], d['total'], True)
#self.last_stats = f"{d['txs']:,d} txs, {d['claims']:,d} claims and {d['supports']:,d} supports"
#self.get_or_create_bar("read", "├─ blocks read", "blocks", d['blocks'], True)
#self.get_or_create_bar("save", "└─┬ txs saved", "txs", d['txs'], True)
else:
if d['done'] == (-1,)*len(d['done']):
base_name = name[:name.rindex('.')]
for child_name, child_bar in self.bars.items():
if child_name.startswith(base_name):
child_bar.close()
bar.close()
self.bars.pop(name)
else:
if len(d['done']) == 2:
bar.update((d['done'][0]-bar.n, d['done'][1]-bar.n2))
else:
bar.update(d['done'][0]-bar.n)
def sync_task(self, name, d):
bar_name = f"{name}#{d['id']}"
bar = self.bars.get(bar_name)
if bar is None:
#assert d['done'][0] == 0
label = d.get('label', name[-11:])
self.get_or_create_bar(
f"{name}#{d['id']}", label, d['units'], d['total'],
name.split('.')[-1] not in ('insert', 'update', 'file')
)
else:
if d['done'][0] != -1:
main_bar_name = f"{name[:name.rindex('.')]}.main"
if len(d['done']) > 1:
diff = tuple(a-b for a, b in zip(d['done'], (bar.n, bar.n2)))
else:
diff = d['done'][0] - bar.n
if main_bar_name != name:
main_bar = self.bars.get(main_bar_name)
if main_bar and main_bar.unit == bar.unit:
main_bar.update(diff)
bar.update(diff)
if d['done'][0] == -1 or d['done'][0] == bar.total:
bar.close()
self.bars.pop(bar_name)
def update_other_bars(self, e, d):
if d['total'] == 0:
return
bar = self.bars.get(e)
if not bar:
name = (
' '.join(e.split('.')[-2:])
.replace('support', 'suprt')
.replace('channels', 'chanls')
.replace('signatures', 'sigs')
)
bar = self.get_or_create_bar(e, f"├─ {name:>12}", d['unit'], d['total'], True)
diff = d['step']-bar.n
bar.update(diff)
#if d['step'] == d['total']:
#bar.close()
def on_sync_progress(self, event):
e, d = event['event'], event.get('data', {})
if e.endswith(".init"):
self.sync_init(e, d)
elif e.endswith(".main"):
self.sync_main(e, d)
else:
self.sync_task(e, d)
# if e.endswith("sync.start"):
# self.sync_start(d)
# self.stderr.flush(self.bars['read'].write)
# elif e.endswith("sync.complete"):
# self.stderr.flush(self.bars['read'].write, True)
# self.sync_complete()
# else:
# self.stderr.flush(self.bars['read'].write)
# self.update_progress(e, d)
def console_class_from_name(name) -> Type[Console]:
return {'basic': Basic, 'advanced': Advanced}.get(name, Console)
+6
View File
@@ -0,0 +1,6 @@
DEFAULT_PAGE_SIZE = 20
NULL_HASH32 = b'\x00'*32
CENT = 1000000
COIN = 100*CENT
File renamed without changes.
+86
View File
@@ -0,0 +1,86 @@
from lbry.crypto.hash import double_sha256
from lbry.crypto.util import bytes_to_int, int_to_bytes
class Base58Error(Exception):
""" Exception used for Base58 errors. """
class Base58:
""" Class providing base 58 functionality. """
chars = '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'
assert len(chars) == 58
char_map = {c: n for n, c in enumerate(chars)}
@classmethod
def char_value(cls, c):
val = cls.char_map.get(c)
if val is None:
raise Base58Error(f'invalid base 58 character "{c}"')
return val
@classmethod
def decode(cls, txt):
""" Decodes txt into a big-endian bytearray. """
if isinstance(txt, memoryview):
txt = str(txt)
if isinstance(txt, bytes):
txt = txt.decode()
if not isinstance(txt, str):
raise TypeError('a string is required')
if not txt:
raise Base58Error('string cannot be empty')
value = 0
for c in txt:
value = value * 58 + cls.char_value(c)
result = int_to_bytes(value)
# Prepend leading zero bytes if necessary
count = 0
for c in txt:
if c != '1':
break
count += 1
if count:
result = bytes((0,)) * count + result
return result
@classmethod
def encode(cls, be_bytes):
"""Converts a big-endian bytearray into a base58 string."""
value = bytes_to_int(be_bytes)
txt = ''
while value:
value, mod = divmod(value, 58)
txt += cls.chars[mod]
for byte in be_bytes:
if byte != 0:
break
txt += '1'
return txt[::-1]
@classmethod
def decode_check(cls, txt, hash_fn=double_sha256):
""" Decodes a Base58Check-encoded string to a payload. The version prefixes it. """
be_bytes = cls.decode(txt)
result, check = be_bytes[:-4], be_bytes[-4:]
if check != hash_fn(result)[:4]:
raise Base58Error(f'invalid base 58 checksum for {txt}')
return result
@classmethod
def encode_check(cls, payload, hash_fn=double_sha256):
""" Encodes a payload bytearray (which includes the version byte(s))
into a Base58Check string."""
be_bytes = payload + hash_fn(payload)[:4]
return cls.encode(be_bytes)
+253
View File
@@ -0,0 +1,253 @@
from coincurve import PublicKey, PrivateKey as _PrivateKey
from lbry.crypto.hash import hmac_sha512, hash160, double_sha256
from lbry.crypto.base58 import Base58
from lbry.utils import cachedproperty
class DerivationError(Exception):
""" Raised when an invalid derivation occurs. """
class _KeyBase:
""" A BIP32 Key, public or private. """
def __init__(self, ledger, chain_code, n, depth, parent):
if not isinstance(chain_code, (bytes, bytearray)):
raise TypeError('chain code must be raw bytes')
if len(chain_code) != 32:
raise ValueError('invalid chain code')
if not 0 <= n < 1 << 32:
raise ValueError('invalid child number')
if not 0 <= depth < 256:
raise ValueError('invalid depth')
if parent is not None:
if not isinstance(parent, type(self)):
raise TypeError('parent key has bad type')
self.ledger = ledger
self.chain_code = chain_code
self.n = n
self.depth = depth
self.parent = parent
def _hmac_sha512(self, msg):
""" Use SHA-512 to provide an HMAC, returned as a pair of 32-byte objects. """
hmac = hmac_sha512(self.chain_code, msg)
return hmac[:32], hmac[32:]
def _extended_key(self, ver_bytes, raw_serkey):
""" Return the 78-byte extended key given prefix version bytes and serialized key bytes. """
if not isinstance(ver_bytes, (bytes, bytearray)):
raise TypeError('ver_bytes must be raw bytes')
if len(ver_bytes) != 4:
raise ValueError('ver_bytes must have length 4')
if not isinstance(raw_serkey, (bytes, bytearray)):
raise TypeError('raw_serkey must be raw bytes')
if len(raw_serkey) != 33:
raise ValueError('raw_serkey must have length 33')
return (ver_bytes + bytes((self.depth,))
+ self.parent_fingerprint() + self.n.to_bytes(4, 'big')
+ self.chain_code + raw_serkey)
def identifier(self):
raise NotImplementedError
def extended_key(self):
raise NotImplementedError
def fingerprint(self):
""" Return the key's fingerprint as 4 bytes. """
return self.identifier()[:4]
def parent_fingerprint(self):
""" Return the parent key's fingerprint as 4 bytes. """
return self.parent.fingerprint() if self.parent else bytes((0,)*4)
def extended_key_string(self):
""" Return an extended key as a base58 string. """
return Base58.encode_check(self.extended_key())
class PubKey(_KeyBase):
""" A BIP32 public key. """
def __init__(self, ledger, pubkey, chain_code, n, depth, parent=None):
super().__init__(ledger, chain_code, n, depth, parent)
if isinstance(pubkey, PublicKey):
self.verifying_key = pubkey
else:
self.verifying_key = self._verifying_key_from_pubkey(pubkey)
@classmethod
def _verifying_key_from_pubkey(cls, pubkey):
""" Converts a 33-byte compressed pubkey into an PublicKey object. """
if not isinstance(pubkey, (bytes, bytearray)):
raise TypeError('pubkey must be raw bytes')
if len(pubkey) != 33:
raise ValueError('pubkey must be 33 bytes')
if pubkey[0] not in (2, 3):
raise ValueError('invalid pubkey prefix byte')
return PublicKey(pubkey)
@cachedproperty
def pubkey_bytes(self):
""" Return the compressed public key as 33 bytes. """
return self.verifying_key.format(True)
@cachedproperty
def address(self):
""" The public key as a P2PKH address. """
return self.ledger.public_key_to_address(self.pubkey_bytes)
def ec_point(self):
return self.verifying_key.point()
def child(self, n: int):
""" Return the derived child extended pubkey at index N. """
if not 0 <= n < (1 << 31):
raise ValueError('invalid BIP32 public key child number')
msg = self.pubkey_bytes + n.to_bytes(4, 'big')
L_b, R_b = self._hmac_sha512(msg) # pylint: disable=invalid-name
derived_key = self.verifying_key.add(L_b)
return PubKey(self.ledger, derived_key, R_b, n, self.depth + 1, self)
def identifier(self):
""" Return the key's identifier as 20 bytes. """
return hash160(self.pubkey_bytes)
def extended_key(self):
""" Return a raw extended public key. """
return self._extended_key(
self.ledger.extended_public_key_prefix,
self.pubkey_bytes
)
class PrivateKey(_KeyBase):
"""A BIP32 private key."""
HARDENED = 1 << 31
def __init__(self, ledger, privkey, chain_code, n, depth, parent=None):
super().__init__(ledger, chain_code, n, depth, parent)
if isinstance(privkey, _PrivateKey):
self.signing_key = privkey
else:
self.signing_key = self._signing_key_from_privkey(privkey)
@classmethod
def _signing_key_from_privkey(cls, private_key):
""" Converts a 32-byte private key into an coincurve.PrivateKey object. """
return _PrivateKey.from_int(PrivateKey._private_key_secret_exponent(private_key))
@classmethod
def _private_key_secret_exponent(cls, private_key):
""" Return the private key as a secret exponent if it is a valid private key. """
if not isinstance(private_key, (bytes, bytearray)):
raise TypeError('private key must be raw bytes')
if len(private_key) != 32:
raise ValueError('private key must be 32 bytes')
return int.from_bytes(private_key, 'big')
@classmethod
def from_seed(cls, ledger, seed):
# This hard-coded message string seems to be coin-independent...
hmac = hmac_sha512(b'Bitcoin seed', seed)
privkey, chain_code = hmac[:32], hmac[32:]
return cls(ledger, privkey, chain_code, 0, 0)
@cachedproperty
def private_key_bytes(self):
""" Return the serialized private key (no leading zero byte). """
return self.signing_key.secret
@cachedproperty
def public_key(self):
""" Return the corresponding extended public key. """
verifying_key = self.signing_key.public_key
parent_pubkey = self.parent.public_key if self.parent else None
return PubKey(self.ledger, verifying_key, self.chain_code, self.n, self.depth,
parent_pubkey)
def ec_point(self):
return self.public_key.ec_point()
def secret_exponent(self):
""" Return the private key as a secret exponent. """
return self.signing_key.to_int()
def wif(self):
""" Return the private key encoded in Wallet Import Format. """
return self.ledger.private_key_to_wif(self.private_key_bytes)
def address(self):
""" The public key as a P2PKH address. """
return self.public_key.address
def child(self, n):
""" Return the derived child extended private key at index N."""
if not 0 <= n < (1 << 32):
raise ValueError('invalid BIP32 private key child number')
if n >= self.HARDENED:
serkey = b'\0' + self.private_key_bytes
else:
serkey = self.public_key.pubkey_bytes
msg = serkey + n.to_bytes(4, 'big')
L_b, R_b = self._hmac_sha512(msg) # pylint: disable=invalid-name
derived_key = self.signing_key.add(L_b)
return PrivateKey(self.ledger, derived_key, R_b, n, self.depth + 1, self)
def sign(self, data):
""" Produce a signature for piece of data by double hashing it and signing the hash. """
return self.signing_key.sign(data, hasher=double_sha256)
def identifier(self):
"""Return the key's identifier as 20 bytes."""
return self.public_key.identifier()
def extended_key(self):
"""Return a raw extended private key."""
return self._extended_key(
self.ledger.extended_private_key_prefix,
b'\0' + self.private_key_bytes
)
def _from_extended_key(ledger, ekey):
"""Return a PubKey or PrivateKey from an extended key raw bytes."""
if not isinstance(ekey, (bytes, bytearray)):
raise TypeError('extended key must be raw bytes')
if len(ekey) != 78:
raise ValueError('extended key must have length 78')
depth = ekey[4]
n = int.from_bytes(ekey[9:13], 'big')
chain_code = ekey[13:45]
if ekey[:4] == ledger.extended_public_key_prefix:
pubkey = ekey[45:]
key = PubKey(ledger, pubkey, chain_code, n, depth)
elif ekey[:4] == ledger.extended_private_key_prefix:
if ekey[45] != 0:
raise ValueError('invalid extended private key prefix byte')
privkey = ekey[46:]
key = PrivateKey(ledger, privkey, chain_code, n, depth)
else:
raise ValueError('version bytes unrecognised')
return key
def from_extended_key_string(ledger, ekey_str):
"""Given an extended key string, such as
xpub6BsnM1W2Y7qLMiuhi7f7dbAwQZ5Cz5gYJCRzTNainXzQXYjFwtuQXHd
3qfi3t3KJtHxshXezfjft93w4UE7BGMtKwhqEHae3ZA7d823DVrL
return a PubKey or PrivateKey.
"""
return _from_extended_key(ledger, Base58.decode_check(ekey_str))
+71
View File
@@ -0,0 +1,71 @@
import os
import base64
import typing
from cryptography.hazmat.primitives.kdf.scrypt import Scrypt
from cryptography.hazmat.primitives.ciphers import Cipher, modes
from cryptography.hazmat.primitives.ciphers.algorithms import AES
from cryptography.hazmat.primitives.padding import PKCS7
from cryptography.hazmat.backends import default_backend
from lbry.error import InvalidPasswordError
from lbry.crypto.hash import double_sha256
def aes_encrypt(secret: str, value: str, init_vector: bytes = None) -> str:
if init_vector is not None:
assert len(init_vector) == 16
else:
init_vector = os.urandom(16)
key = double_sha256(secret.encode())
encryptor = Cipher(AES(key), modes.CBC(init_vector), default_backend()).encryptor()
padder = PKCS7(AES.block_size).padder()
padded_data = padder.update(value.encode()) + padder.finalize()
encrypted_data = encryptor.update(padded_data) + encryptor.finalize()
return base64.b64encode(init_vector + encrypted_data).decode()
def aes_decrypt(secret: str, value: str) -> typing.Tuple[str, bytes]:
try:
data = base64.b64decode(value.encode())
key = double_sha256(secret.encode())
init_vector, data = data[:16], data[16:]
decryptor = Cipher(AES(key), modes.CBC(init_vector), default_backend()).decryptor()
unpadder = PKCS7(AES.block_size).unpadder()
result = unpadder.update(decryptor.update(data)) + unpadder.finalize()
return result.decode(), init_vector
except UnicodeDecodeError:
raise InvalidPasswordError()
except ValueError as e:
if e.args[0] == 'Invalid padding bytes.':
raise InvalidPasswordError()
raise
def better_aes_encrypt(secret: str, value: bytes) -> bytes:
init_vector = os.urandom(16)
key = scrypt(secret.encode(), salt=init_vector)
encryptor = Cipher(AES(key), modes.CBC(init_vector), default_backend()).encryptor()
padder = PKCS7(AES.block_size).padder()
padded_data = padder.update(value) + padder.finalize()
encrypted_data = encryptor.update(padded_data) + encryptor.finalize()
return base64.b64encode(b's:8192:16:1:' + init_vector + encrypted_data)
def better_aes_decrypt(secret: str, value: bytes) -> bytes:
try:
data = base64.b64decode(value)
_, scryp_n, scrypt_r, scrypt_p, data = data.split(b':', maxsplit=4)
init_vector, data = data[:16], data[16:]
key = scrypt(secret.encode(), init_vector, int(scryp_n), int(scrypt_r), int(scrypt_p))
decryptor = Cipher(AES(key), modes.CBC(init_vector), default_backend()).decryptor()
unpadder = PKCS7(AES.block_size).unpadder()
return unpadder.update(decryptor.update(data)) + unpadder.finalize()
except ValueError as e:
if e.args[0] == 'Invalid padding bytes.':
raise InvalidPasswordError()
raise
def scrypt(passphrase, salt, scrypt_n=1<<13, scrypt_r=16, scrypt_p=1):
kdf = Scrypt(salt, length=32, n=scrypt_n, r=scrypt_r, p=scrypt_p, backend=default_backend())
return kdf.derive(passphrase)
+47
View File
@@ -0,0 +1,47 @@
import hashlib
import hmac
from binascii import hexlify, unhexlify
def sha256(x):
""" Simple wrapper of hashlib sha256. """
return hashlib.sha256(x).digest()
def sha512(x):
""" Simple wrapper of hashlib sha512. """
return hashlib.sha512(x).digest()
def ripemd160(x):
""" Simple wrapper of hashlib ripemd160. """
h = hashlib.new('ripemd160')
h.update(x)
return h.digest()
def double_sha256(x):
""" SHA-256 of SHA-256, as used extensively in bitcoin. """
return sha256(sha256(x))
def hmac_sha512(key, msg):
""" Use SHA-512 to provide an HMAC. """
return hmac.new(key, msg, hashlib.sha512).digest()
def hash160(x):
""" RIPEMD-160 of SHA-256.
Used to make bitcoin addresses from pubkeys. """
return ripemd160(sha256(x))
def hash_to_hex_str(x: bytes) -> str:
""" Convert a big-endian binary hash to displayed hex string.
Display form of a binary hash is reversed and converted to hex. """
return hexlify(x[::-1])
def hex_str_to_hash(x: str) -> bytes:
""" Convert a displayed hex string to a binary hash. """
return unhexlify(x)[::-1]
+13
View File
@@ -0,0 +1,13 @@
from binascii import unhexlify, hexlify
def bytes_to_int(be_bytes):
""" Interprets a big-endian sequence of bytes as an integer. """
return int(hexlify(be_bytes), 16)
def int_to_bytes(value):
""" Converts an integer to a big-endian sequence of bytes. """
length = (value.bit_length() + 7) // 8
s = '%x' % value
return unhexlify(('0' * (len(s) % 2) + s).zfill(length * 2))
+5
View File
@@ -0,0 +1,5 @@
from .database import Database, Result
from .constants import (
TXO_TYPES, SPENDABLE_TYPE_CODES,
CLAIM_TYPE_CODES, CLAIM_TYPE_NAMES
)
+73
View File
@@ -0,0 +1,73 @@
MAX_QUERY_VARIABLES = 900
TXO_TYPES = {
"other": 0,
"stream": 1,
"channel": 2,
"support": 3,
"purchase": 4,
"collection": 5,
"repost": 6,
}
CLAIM_TYPE_NAMES = [
'stream',
'channel',
'collection',
'repost',
]
CONTENT_TYPE_NAMES = [
name for name in CLAIM_TYPE_NAMES if name != "channel"
]
CLAIM_TYPE_CODES = [
TXO_TYPES[name] for name in CLAIM_TYPE_NAMES
]
CONTENT_TYPE_CODES = [
TXO_TYPES[name] for name in CONTENT_TYPE_NAMES
]
SPENDABLE_TYPE_CODES = [
TXO_TYPES['other'],
TXO_TYPES['purchase']
]
STREAM_TYPES = {
'video': 1,
'audio': 2,
'image': 3,
'document': 4,
'binary': 5,
'model': 6
}
MATURE_TAGS = (
'nsfw', 'porn', 'xxx', 'mature', 'adult', 'sex'
)
ATTRIBUTE_ARRAY_MAX_LENGTH = 100
SEARCH_INTEGER_PARAMS = {
'height', 'creation_height', 'activation_height', 'expiration_height',
'timestamp', 'creation_timestamp', 'duration', 'release_time', 'fee_amount',
'tx_position', 'channel_join', 'reposted',
'amount', 'staked_amount', 'support_amount',
'trend_group', 'trend_mixed', 'trend_local', 'trend_global',
}
SEARCH_PARAMS = {
'name', 'text', 'claim_id', 'claim_ids', 'txid', 'nout', 'channel', 'channel_ids', 'not_channel_ids',
'public_key_id', 'claim_type', 'stream_types', 'media_types', 'fee_currency',
'has_channel_signature', 'signature_valid',
'any_tags', 'all_tags', 'not_tags', 'reposted_claim_id',
'any_locations', 'all_locations', 'not_locations',
'any_languages', 'all_languages', 'not_languages',
'is_controlling', 'limit', 'offset', 'order_by',
'no_totals',
} | SEARCH_INTEGER_PARAMS
SEARCH_ORDER_FIELDS = {
'name', 'claim_hash', 'claim_id'
} | SEARCH_INTEGER_PARAMS
+369
View File
@@ -0,0 +1,369 @@
import os
import asyncio
import tempfile
import multiprocessing as mp
from typing import List, Optional, Iterable, Iterator, TypeVar, Generic, TYPE_CHECKING, Dict, Tuple
from concurrent.futures import Executor, ThreadPoolExecutor, ProcessPoolExecutor
from functools import partial
from sqlalchemy import create_engine, text
from lbry.event import EventController
from lbry.crypto.bip32 import PubKey
from lbry.blockchain.transaction import Transaction, Output
from .constants import TXO_TYPES, CLAIM_TYPE_CODES
from .query_context import initialize, uninitialize, ProgressPublisher
from . import queries as q
from . import sync
if TYPE_CHECKING:
from lbry.blockchain.ledger import Ledger
def clean_wallet_account_ids(constraints):
wallet = constraints.pop('wallet', None)
account = constraints.pop('account', None)
accounts = constraints.pop('accounts', [])
if account and not accounts:
accounts = [account]
if wallet:
constraints['wallet_account_ids'] = [account.id for account in wallet.accounts]
if not accounts:
accounts = wallet.accounts
if accounts:
constraints['account_ids'] = [account.id for account in accounts]
async def add_channel_keys_to_txo_results(accounts: List, txos: Iterable[Output]):
sub_channels = set()
for txo in txos:
if txo.is_claim and txo.claim.is_channel:
for account in accounts:
private_key = await account.get_channel_private_key(
txo.claim.channel.public_key_bytes
)
if private_key:
txo.private_key = private_key
break
if txo.channel is not None:
sub_channels.add(txo.channel)
if sub_channels:
await add_channel_keys_to_txo_results(accounts, sub_channels)
ResultType = TypeVar('ResultType')
class Result(Generic[ResultType]):
__slots__ = 'rows', 'total', 'censor'
def __init__(self, rows: List[ResultType], total, censor=None):
self.rows = rows
self.total = total
self.censor = censor
def __getitem__(self, item: int) -> ResultType:
return self.rows[item]
def __iter__(self) -> Iterator[ResultType]:
return iter(self.rows)
def __len__(self):
return len(self.rows)
def __repr__(self):
return repr(self.rows)
class Database:
def __init__(self, ledger: 'Ledger'):
self.url = ledger.conf.db_url_or_default
self.ledger = ledger
self.workers = self._normalize_worker_processes(ledger.conf.workers)
self.executor: Optional[Executor] = None
self.message_queue = mp.Queue()
self.stop_event = mp.Event()
self._on_progress_controller = EventController()
self.on_progress = self._on_progress_controller.stream
self.progress_publisher = ProgressPublisher(
self.message_queue, self._on_progress_controller
)
@staticmethod
def _normalize_worker_processes(workers):
if workers == 0:
return os.cpu_count()
elif workers > 0:
return workers
return 1
@classmethod
def temp_from_url_regtest(cls, db_url, lbrycrd_config=None):
from lbry import Config, RegTestLedger # pylint: disable=import-outside-toplevel
directory = tempfile.mkdtemp()
if lbrycrd_config:
conf = lbrycrd_config
conf.data_dir = directory
conf.download_dir = directory
conf.wallet_dir = directory
else:
conf = Config.with_same_dir(directory)
conf.set(blockchain="regtest", db_url=db_url)
ledger = RegTestLedger(conf)
return cls(ledger)
@classmethod
def temp_sqlite_regtest(cls, lbrycrd_config=None):
from lbry import Config, RegTestLedger # pylint: disable=import-outside-toplevel
directory = tempfile.mkdtemp()
if lbrycrd_config:
conf = lbrycrd_config
conf.data_dir = directory
conf.download_dir = directory
conf.wallet_dir = directory
else:
conf = Config.with_same_dir(directory).set(blockchain="regtest")
ledger = RegTestLedger(conf)
return cls(ledger)
@classmethod
def temp_sqlite(cls):
from lbry import Config, Ledger # pylint: disable=import-outside-toplevel
conf = Config.with_same_dir(tempfile.mkdtemp())
return cls(Ledger(conf))
@classmethod
def from_url(cls, db_url):
from lbry import Config, Ledger # pylint: disable=import-outside-toplevel
return cls(Ledger(Config.with_null_dir().set(db_url=db_url)))
@classmethod
def in_memory(cls):
return cls.from_url('sqlite:///:memory:')
def sync_create(self, name):
engine = create_engine(self.url)
db = engine.connect()
db.execute(text("COMMIT"))
db.execute(text(f"CREATE DATABASE {name}"))
async def create(self, name):
return await asyncio.get_running_loop().run_in_executor(None, self.sync_create, name)
def sync_drop(self, name):
engine = create_engine(self.url)
db = engine.connect()
db.execute(text("COMMIT"))
db.execute(text(f"DROP DATABASE IF EXISTS {name}"))
async def drop(self, name):
return await asyncio.get_running_loop().run_in_executor(None, self.sync_drop, name)
async def open(self):
assert self.executor is None, "Database already open."
self.progress_publisher.start()
kwargs = {
"initializer": initialize,
"initargs": (
self.ledger,
self.message_queue, self.stop_event
)
}
if self.workers > 1:
self.executor = ProcessPoolExecutor(max_workers=self.workers, **kwargs)
else:
self.executor = ThreadPoolExecutor(max_workers=1, **kwargs)
return await self.run(q.check_version_and_create_tables)
async def close(self):
self.progress_publisher.stop()
if self.executor is not None:
if isinstance(self.executor, ThreadPoolExecutor):
await self.run(uninitialize)
self.executor.shutdown()
self.executor = None
# fixes "OSError: handle is closed"
# seems to only happen when running in PyCharm
# https://github.com/python/cpython/pull/6084#issuecomment-564585446
# TODO: delete this in Python 3.8/3.9?
from concurrent.futures.process import _threads_wakeups # pylint: disable=import-outside-toplevel
_threads_wakeups.clear()
async def run(self, func, *args, **kwargs):
if kwargs:
clean_wallet_account_ids(kwargs)
return await asyncio.get_running_loop().run_in_executor(
self.executor, partial(func, *args, **kwargs)
)
async def fetch_result(self, func, *args, **kwargs) -> Result:
rows, total = await self.run(func, *args, **kwargs)
return Result(rows, total)
async def execute(self, sql):
return await self.run(q.execute, sql)
async def execute_sql_object(self, sql):
return await self.run(q.execute_sql_object, sql)
async def execute_fetchall(self, sql):
return await self.run(q.execute_fetchall, sql)
async def has_filters(self):
return await self.run(q.has_filters)
async def has_claims(self):
return await self.run(q.has_claims)
async def has_supports(self):
return await self.run(q.has_supports)
async def has_wallet(self, wallet_id):
return await self.run(q.has_wallet, wallet_id)
async def get_wallet(self, wallet_id: str):
return await self.run(q.get_wallet, wallet_id)
async def add_wallet(self, wallet_id: str, data: str):
return await self.run(q.add_wallet, wallet_id, data)
async def get_best_block_height(self) -> int:
return await self.run(q.get_best_block_height)
async def process_all_things_after_sync(self):
return await self.run(sync.process_all_things_after_sync)
async def get_block_headers(self, start_height: int, end_height: int = None):
return await self.run(q.get_block_headers, start_height, end_height)
async def get_filters(self, start_height: int, end_height: int = None, granularity: int = 0):
return await self.run(q.get_filters, start_height, end_height, granularity)
async def insert_block(self, block):
return await self.run(q.insert_block, block)
async def insert_transaction(self, block_hash, tx):
return await self.run(q.insert_transaction, block_hash, tx)
async def update_address_used_times(self, addresses):
return await self.run(q.update_address_used_times, addresses)
async def reserve_outputs(self, txos, is_reserved=True):
txo_hashes = [txo.hash for txo in txos]
if txo_hashes:
return await self.run(
q.reserve_outputs, txo_hashes, is_reserved
)
async def release_outputs(self, txos):
return await self.reserve_outputs(txos, is_reserved=False)
async def release_tx(self, tx):
return await self.release_outputs([txi.txo_ref.txo for txi in tx.inputs])
async def release_all_outputs(self, account):
return await self.run(q.release_all_outputs, account.id)
async def get_balance(self, **constraints):
return await self.run(q.get_balance, **constraints)
async def get_report(self, accounts):
return await self.run(q.get_report, accounts=accounts)
async def get_addresses(self, **constraints) -> Result[dict]:
addresses = await self.fetch_result(q.get_addresses, **constraints)
if addresses and 'pubkey' in addresses[0]:
for address in addresses:
address['pubkey'] = PubKey(
self.ledger, bytes(address.pop('pubkey')), bytes(address.pop('chain_code')),
address.pop('n'), address.pop('depth')
)
return addresses
async def get_all_addresses(self):
return await self.run(q.get_all_addresses)
async def get_address(self, **constraints):
for address in await self.get_addresses(limit=1, **constraints):
return address
async def add_keys(self, account, chain, pubkeys):
return await self.run(q.add_keys, [{
'account': account.id,
'address': k.address,
'chain': chain,
'pubkey': k.pubkey_bytes,
'chain_code': k.chain_code,
'n': k.n,
'depth': k.depth
} for k in pubkeys])
async def get_transactions(self, **constraints) -> Result[Transaction]:
return await self.fetch_result(q.get_transactions, **constraints)
async def get_transaction(self, **constraints) -> Optional[Transaction]:
txs = await self.get_transactions(limit=1, **constraints)
if txs:
return txs[0]
async def get_purchases(self, **constraints) -> Result[Output]:
return await self.fetch_result(q.get_purchases, **constraints)
async def search_claims(self, **constraints) -> Result[Output]:
#assert set(constraints).issubset(SEARCH_PARAMS), \
# f"Search query contains invalid arguments: {set(constraints).difference(SEARCH_PARAMS)}"
claims, total, censor = await self.run(q.search_claims, **constraints)
return Result(claims, total, censor)
async def protobuf_search_claims(self, **constraints) -> str:
return await self.run(q.protobuf_search_claims, **constraints)
async def search_supports(self, **constraints) -> Result[Output]:
return await self.fetch_result(q.search_supports, **constraints)
async def sum_supports(self, claim_hash, include_channel_content=False, exclude_own_supports=False) \
-> Tuple[List[Dict], int]:
return await self.run(q.sum_supports, claim_hash, include_channel_content, exclude_own_supports)
async def resolve(self, urls, **kwargs) -> Dict[str, Output]:
return await self.run(q.resolve, urls, **kwargs)
async def protobuf_resolve(self, urls, **kwargs) -> str:
return await self.run(q.protobuf_resolve, urls, **kwargs)
async def get_txo_sum(self, **constraints) -> int:
return await self.run(q.get_txo_sum, **constraints)
async def get_txo_plot(self, **constraints) -> List[dict]:
return await self.run(q.get_txo_plot, **constraints)
async def get_txos(self, **constraints) -> Result[Output]:
txos = await self.fetch_result(q.get_txos, **constraints)
if 'wallet' in constraints:
await add_channel_keys_to_txo_results(constraints['wallet'].accounts, txos)
return txos
async def get_utxos(self, **constraints) -> Result[Output]:
return await self.get_txos(spent_height=0, **constraints)
async def get_supports(self, **constraints) -> Result[Output]:
return await self.get_utxos(txo_type=TXO_TYPES['support'], **constraints)
async def get_claims(self, **constraints) -> Result[Output]:
if 'txo_type' not in constraints:
constraints['txo_type__in'] = CLAIM_TYPE_CODES
txos = await self.fetch_result(q.get_txos, **constraints)
if 'wallet' in constraints:
await add_channel_keys_to_txo_results(constraints['wallet'].accounts, txos)
return txos
async def get_streams(self, **constraints) -> Result[Output]:
return await self.get_claims(txo_type=TXO_TYPES['stream'], **constraints)
async def get_channels(self, **constraints) -> Result[Output]:
return await self.get_claims(txo_type=TXO_TYPES['channel'], **constraints)
async def get_collections(self, **constraints) -> Result[Output]:
return await self.get_claims(txo_type=TXO_TYPES['collection'], **constraints)
Loaded 100 of 521 files, more files were not shown because too many files have changed in this diff. Show more