Compare commits

..
169 changed files with 8692 additions and 15060 deletions

No files matched your search

-3
View File
@@ -1,4 +1,3 @@
.DS_Store
/build
/dist
/.tox
@@ -9,5 +8,3 @@
lbrynet.egg-info
__pycache__
_trial_temp/
/tests/integration/files
+2 -13
View File
@@ -35,17 +35,6 @@ jobs:
- coverage combine tests/
- bash <(curl -s https://codecov.io/bash)
- name: "Run Examples"
install:
- pip install coverage
- pip install git+https://github.com/lbryio/torba.git#egg=torba
- pip install -e .
script:
- HOME=/tmp coverage run -p --source=lbrynet scripts/generate_json_api.py
after_success:
- coverage combine
- bash <(curl -s https://codecov.io/bash)
- stage: build
name: "Windows"
language: generic
@@ -80,7 +69,7 @@ jobs:
install:
- pip3 install pyinstaller
- pip3 install git+https://github.com/lbryio/torba.git
- python3 scripts/set_build.py
- python scripts/set_build.py
- pip3 install -e .
script:
- pyinstaller -F -n lbrynet lbrynet/extras/cli.py
@@ -123,7 +112,7 @@ jobs:
- echo "$DOCKER_PASSWORD" | docker login --username "$DOCKER_USERNAME" --password-stdin
- travis_retry docker build -t lbry/wallet-server:$TRAVIS_TAG -f scripts/Dockerfile.wallet_server .
- docker push lbry/wallet-server:$TRAVIS_TAG
- if: tag IS blank AND branch = master AND NOT type IN (pull_request)
- if: tag IS blank AND branch = master
stage: build
name: "Wallet Server Docker Image - Master"
script:
+1 -1
View File
@@ -1,3 +1,3 @@
## Contributing to LBRY
https://lbry.tech/contribute
https://lbry.io/faq/contributing
+12 -11
View File
@@ -13,6 +13,15 @@ Running `lbrynet` from source requires Python 3.6 or higher (3.7 is preferred).
After installing python 3, you'll need to install some additional libraries depending on your operating system.
### Virtualenv
Once python 3 is installed run `python3 -m pip install virtualenv` to install virtualenv.
### Windows
Windows users will need to install `Visual C++ Build Tools`, which can be installed by [Microsoft Build Tools](Microsoft Build Tools 2015)
### macOS
macOS users will need to install [xcode command line tools](https://developer.xamarin.com/guides/testcloud/calabash/configuring/osx/install-xcode-command-line-tools/) and [homebrew](http://brew.sh/).
@@ -20,17 +29,15 @@ macOS users will need to install [xcode command line tools](https://developer.xa
Remaining dependencies can then be installed by running:
```
brew install python protobuf
brew install python3 protobuf
```
Assistance installing Python3: https://docs.python-guide.org/starting/install3/osx/
### Linux
On Ubuntu (we recommend 18.04), install the following:
```
sudo apt-get install build-essential python3.7 python3.7-dev git python3-venv libssl-dev python-protobuf
sudo apt-get install build-essential python3.7 python3.7-dev git python-virtualenv libssl-dev python-protobuf
```
On Raspbian, you will also need to install `python-pyparsing`.
@@ -45,14 +52,8 @@ To install:
git clone https://github.com/lbryio/lbry.git
cd lbry
Creating venv:
python -m venv lbry-venv
Activating lbry-venv on Linux/Mac:
virtualenv lbry-venv --python=python3.7
source lbry-venv/bin/activate
Activating lbry-venv on Windows:
lbry-venv\Scripts\activate
python --version # Python 2 is not supported. Make sure you're on Python 3.7
+4 -4
View File
@@ -8,7 +8,7 @@ LBRY SDK for Python is currently the most full featured implementation of the LB
* Kademlia DHT (Distributed Hash Table) implementation for finding peers to download from and announcing to peers what we have to host ([lbrynet.dht](https://github.com/lbryio/lbry/tree/master/lbrynet/dht)).
* Blob exchange protocol for transferring encrypted blobs of content and negotiating payments ([lbrynet.blob_exchange](https://github.com/lbryio/lbry/tree/master/lbrynet/blob_exchange)).
* Protobuf schema for encoding and decoding metadata stored on the blockchain ([lbrynet.schema](https://github.com/lbryio/lbry/tree/master/lbrynet/schema)).
* Wallet implementation for the LBRY blockchain ([lbrynet.wallet](https://github.com/lbryio/lbry/tree/master/lbrynet/wallet)).
* Wallet implementation for the LBRY blockchain ([lbrynet.extras.wallet](https://github.com/lbryio/lbry/tree/master/lbrynet/extras/wallet)).
* Daemon with a JSON-RPC API to ease building end user applications in any language and for automating various tasks ([lbrynet.extras.daemon](https://github.com/lbryio/lbry/tree/master/lbrynet/extras/daemon)).
## Installation
@@ -33,7 +33,7 @@ Installing from source is also relatively painless. Full instructions are in [IN
## Contributing
Contributions to this project are welcome, encouraged, and compensated. For more details, please check [this](https://lbry.tech/contribute) link.
Contributions to this project are welcome, encouraged, and compensated. For more details, please check [this](https://lbry.io/faq/contributing) link.
## License
@@ -41,11 +41,11 @@ This project is MIT licensed. For the full license, see [LICENSE](LICENSE).
## Security
We take security seriously. Please contact security@lbry.com regarding any security issues. [Our GPG key is here](https://lbry.io/faq/gpg-key) if you need it.
We take security seriously. Please contact security@lbry.io regarding any security issues. [Our GPG key is here](https://lbry.io/faq/gpg-key) if you need it.
## Contact
The primary contact for this project is [@eukreign](mailto:lex@lbry.com).
The primary contact for this project is [@eukreign](mailto:lex@lbry.io).
## Additional information and links
+1323 -2625
View File
File diff suppressed because it is too large. Load diff
+3 -4
View File
@@ -12,13 +12,12 @@ blockchain_name: lbrycrd_main
data_dir: /home/lbry/.lbrynet
download_directory: /home/lbry/downloads
save_blobs: true
save_files: false
delete_blobs_on_remove: True
dht_node_port: 4444
peer_port: 3333
use_upnp: true
use_upnp: True
#components_to_skip:
# - peer_protocol_server
# - hash_announcer
# - blob_server
# - dht
+1 -1
View File
@@ -1,4 +1,4 @@
__name__ = "lbrynet"
__version__ = "0.37.0rc6"
__version__ = "0.34.0"
version = tuple(__version__.split('.'))
+130 -268
View File
@@ -4,8 +4,6 @@ import asyncio
import binascii
import logging
import typing
import contextlib
from io import BytesIO
from cryptography.hazmat.primitives.ciphers import Cipher, modes
from cryptography.hazmat.primitives.ciphers.algorithms import AES
from cryptography.hazmat.primitives.padding import PKCS7
@@ -23,6 +21,10 @@ log = logging.getLogger(__name__)
_hexmatch = re.compile("^[a-f,0-9]+$")
def is_valid_hashcharacter(char: str) -> bool:
return len(char) == 1 and _hexmatch.match(char)
def is_valid_blobhash(blobhash: str) -> bool:
"""Checks whether the blobhash is the correct length and contains only
valid characters (0-9, a-f)
@@ -44,74 +46,143 @@ def encrypt_blob_bytes(key: bytes, iv: bytes, unencrypted: bytes) -> typing.Tupl
return encrypted, digest.hexdigest()
def decrypt_blob_bytes(data: bytes, length: int, key: bytes, iv: bytes) -> bytes:
if len(data) != length:
raise ValueError("unexpected length")
cipher = Cipher(AES(key), modes.CBC(iv), backend=backend)
unpadder = PKCS7(AES.block_size).unpadder()
decryptor = cipher.decryptor()
return unpadder.update(decryptor.update(data) + decryptor.finalize()) + unpadder.finalize()
class AbstractBlob:
class BlobFile:
"""
A chunk of data (up to 2MB) available on the network which is specified by a sha384 hash
A chunk of data available on the network which is specified by a hashsum
This class is non-io specific
This class is used to create blobs on the local filesystem
when we already know the blob hash before hand (i.e., when downloading blobs)
Also can be used for reading from blobs on the local filesystem
"""
__slots__ = [
'loop',
'blob_hash',
'length',
'blob_completed_callback',
'blob_directory',
'writers',
'verified',
'writing',
'readers'
]
def __init__(self, loop: asyncio.BaseEventLoop, blob_hash: str, length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], asyncio.Task]] = None,
blob_directory: typing.Optional[str] = None):
def __init__(self, loop: asyncio.BaseEventLoop, blob_dir: str, blob_hash: str,
length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['BlobFile'], typing.Awaitable]] = None):
if not is_valid_blobhash(blob_hash):
raise InvalidBlobHashError(blob_hash)
self.loop = loop
self.blob_hash = blob_hash
self.length = length
self.blob_completed_callback = blob_completed_callback
self.blob_directory = blob_directory
self.writers: typing.Dict[typing.Tuple[typing.Optional[str], typing.Optional[int]], HashBlobWriter] = {}
self.blob_dir = blob_dir
self.file_path = os.path.join(blob_dir, self.blob_hash)
self.writers: typing.List[HashBlobWriter] = []
self.verified: asyncio.Event = asyncio.Event(loop=self.loop)
self.writing: asyncio.Event = asyncio.Event(loop=self.loop)
self.readers: typing.List[typing.BinaryIO] = []
self.finished_writing = asyncio.Event(loop=loop)
self.blob_write_lock = asyncio.Lock(loop=loop)
if self.file_exists:
length = int(os.stat(os.path.join(blob_dir, blob_hash)).st_size)
self.length = length
self.verified.set()
self.finished_writing.set()
self.saved_verified_blob = False
self.blob_completed_callback = blob_completed_callback
if not is_valid_blobhash(blob_hash):
raise InvalidBlobHashError(blob_hash)
@property
def file_exists(self):
return os.path.isfile(self.file_path)
def __del__(self):
if self.writers or self.readers:
log.warning("%s not closed before being garbage collected", self.blob_hash)
self.close()
@contextlib.contextmanager
def _reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
raise NotImplementedError()
@contextlib.contextmanager
def reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
if not self.is_readable():
raise OSError(f"{str(type(self))} not readable, {len(self.readers)} readers {len(self.writers)} writers")
with self._reader_context() as reader:
def writer_finished(self, writer: HashBlobWriter):
def callback(finished: asyncio.Future):
try:
self.readers.append(reader)
yield reader
finally:
if reader in self.readers:
self.readers.remove(reader)
error = finished.exception()
except Exception as err:
error = err
if writer in self.writers: # remove this download attempt
self.writers.remove(writer)
if not error: # the blob downloaded, cancel all the other download attempts and set the result
while self.writers:
other = self.writers.pop()
other.finished.cancel()
t = self.loop.create_task(self.save_verified_blob(writer, finished.result()))
t.add_done_callback(lambda *_: self.finished_writing.set())
return
if isinstance(error, (InvalidBlobHashError, InvalidDataError)):
log.debug("writer error downloading %s: %s", self.blob_hash[:8], str(error))
elif not isinstance(error, (DownloadCancelledError, asyncio.CancelledError, asyncio.TimeoutError)):
log.exception("something else")
raise error
return callback
def _write_blob(self, blob_bytes: bytes):
raise NotImplementedError()
async def save_verified_blob(self, writer, verified_bytes: bytes):
def _save_verified():
# log.debug(f"write blob file {self.blob_hash[:8]} from {writer.peer.address}")
if not self.saved_verified_blob and not os.path.isfile(self.file_path):
if self.get_length() == len(verified_bytes):
with open(self.file_path, 'wb') as write_handle:
write_handle.write(verified_bytes)
self.saved_verified_blob = True
else:
raise Exception("length mismatch")
def set_length(self, length) -> None:
async with self.blob_write_lock:
if self.verified.is_set():
return
await self.loop.run_in_executor(None, _save_verified)
if self.blob_completed_callback:
await self.blob_completed_callback(self)
self.verified.set()
def open_for_writing(self) -> HashBlobWriter:
if self.file_exists:
raise OSError(f"File already exists '{self.file_path}'")
fut = asyncio.Future(loop=self.loop)
writer = HashBlobWriter(self.blob_hash, self.get_length, fut)
self.writers.append(writer)
fut.add_done_callback(self.writer_finished(writer))
return writer
async def sendfile(self, writer: asyncio.StreamWriter) -> int:
"""
Read and send the file to the writer and return the number of bytes sent
"""
with open(self.file_path, 'rb') as handle:
return await self.loop.sendfile(writer.transport, handle, count=self.get_length())
def close(self):
while self.writers:
self.writers.pop().finished.cancel()
def delete(self):
self.close()
self.saved_verified_blob = False
if os.path.isfile(self.file_path):
os.remove(self.file_path)
self.verified.clear()
self.finished_writing.clear()
self.length = None
def decrypt(self, key: bytes, iv: bytes) -> bytes:
"""
Decrypt a BlobFile to plaintext bytes
"""
with open(self.file_path, "rb") as f:
buff = f.read()
if len(buff) != self.length:
raise ValueError("unexpected length")
cipher = Cipher(AES(key), modes.CBC(iv), backend=backend)
unpadder = PKCS7(AES.block_size).unpadder()
decryptor = cipher.decryptor()
return unpadder.update(decryptor.update(buff) + decryptor.finalize()) + unpadder.finalize()
@classmethod
async def create_from_unencrypted(cls, loop: asyncio.BaseEventLoop, blob_dir: str, key: bytes,
iv: bytes, unencrypted: bytes, blob_num: int) -> BlobInfo:
"""
Create an encrypted BlobFile from plaintext bytes
"""
blob_bytes, blob_hash = encrypt_blob_bytes(key, iv, unencrypted)
length = len(blob_bytes)
blob = cls(loop, blob_dir, blob_hash, length)
writer = blob.open_for_writing()
writer.write(blob_bytes)
await blob.verified.wait()
return BlobInfo(blob_num, length, binascii.hexlify(iv).decode(), blob_hash)
def set_length(self, length):
if self.length is not None and length == self.length:
return
if self.length is None and 0 <= length <= MAX_BLOB_SIZE:
@@ -119,217 +190,8 @@ class AbstractBlob:
return
log.warning("Got an invalid length. Previous length: %s, Invalid length: %s", self.length, length)
def get_length(self) -> typing.Optional[int]:
def get_length(self):
return self.length
def get_is_verified(self) -> bool:
def get_is_verified(self):
return self.verified.is_set()
def is_readable(self) -> bool:
return self.verified.is_set()
def is_writeable(self) -> bool:
return not self.writing.is_set()
def write_blob(self, blob_bytes: bytes):
if not self.is_writeable():
raise OSError("cannot open blob for writing")
try:
self.writing.set()
self._write_blob(blob_bytes)
finally:
self.writing.clear()
def close(self):
while self.writers:
peer, writer = self.writers.popitem()
if writer and writer.finished and not writer.finished.done() and not self.loop.is_closed():
writer.finished.cancel()
while self.readers:
reader = self.readers.pop()
if reader:
reader.close()
def delete(self):
self.close()
self.verified.clear()
self.length = None
async def sendfile(self, writer: asyncio.StreamWriter) -> int:
"""
Read and send the file to the writer and return the number of bytes sent
"""
if not self.is_readable():
raise OSError('blob files cannot be read')
with self.reader_context() as handle:
return await self.loop.sendfile(writer.transport, handle, count=self.get_length())
def decrypt(self, key: bytes, iv: bytes) -> bytes:
"""
Decrypt a BlobFile to plaintext bytes
"""
with self.reader_context() as reader:
return decrypt_blob_bytes(reader.read(), self.length, key, iv)
@classmethod
async def create_from_unencrypted(
cls, loop: asyncio.BaseEventLoop, blob_dir: typing.Optional[str], key: bytes, iv: bytes,
unencrypted: bytes, blob_num: int,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], None]] = None) -> BlobInfo:
"""
Create an encrypted BlobFile from plaintext bytes
"""
blob_bytes, blob_hash = encrypt_blob_bytes(key, iv, unencrypted)
length = len(blob_bytes)
blob = cls(loop, blob_hash, length, blob_completed_callback, blob_dir)
writer = blob.get_blob_writer()
writer.write(blob_bytes)
await blob.verified.wait()
return BlobInfo(blob_num, length, binascii.hexlify(iv).decode(), blob_hash)
def save_verified_blob(self, verified_bytes: bytes):
if self.verified.is_set():
return
if self.is_writeable():
self._write_blob(verified_bytes)
self.verified.set()
if self.blob_completed_callback:
self.blob_completed_callback(self)
def get_blob_writer(self, peer_address: typing.Optional[str] = None,
peer_port: typing.Optional[int] = None) -> HashBlobWriter:
if (peer_address, peer_port) in self.writers and not self.writers[(peer_address, peer_port)].closed():
raise OSError(f"attempted to download blob twice from {peer_address}:{peer_port}")
fut = asyncio.Future(loop=self.loop)
writer = HashBlobWriter(self.blob_hash, self.get_length, fut)
self.writers[(peer_address, peer_port)] = writer
def remove_writer(_):
if (peer_address, peer_port) in self.writers:
del self.writers[(peer_address, peer_port)]
fut.add_done_callback(remove_writer)
def writer_finished_callback(finished: asyncio.Future):
try:
err = finished.exception()
if err:
raise err
verified_bytes = finished.result()
while self.writers:
_, other = self.writers.popitem()
if other is not writer:
other.close_handle()
self.save_verified_blob(verified_bytes)
except (InvalidBlobHashError, InvalidDataError) as error:
log.warning("writer error downloading %s: %s", self.blob_hash[:8], str(error))
except (DownloadCancelledError, asyncio.CancelledError, asyncio.TimeoutError):
pass
fut.add_done_callback(writer_finished_callback)
return writer
class BlobBuffer(AbstractBlob):
"""
An in-memory only blob
"""
def __init__(self, loop: asyncio.BaseEventLoop, blob_hash: str, length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], asyncio.Task]] = None,
blob_directory: typing.Optional[str] = None):
self._verified_bytes: typing.Optional[BytesIO] = None
super().__init__(loop, blob_hash, length, blob_completed_callback, blob_directory)
@contextlib.contextmanager
def _reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
if not self.is_readable():
raise OSError("cannot open blob for reading")
try:
yield self._verified_bytes
finally:
if self._verified_bytes:
self._verified_bytes.close()
self._verified_bytes = None
self.verified.clear()
def _write_blob(self, blob_bytes: bytes):
if self._verified_bytes:
raise OSError("already have bytes for blob")
self._verified_bytes = BytesIO(blob_bytes)
def delete(self):
if self._verified_bytes:
self._verified_bytes.close()
self._verified_bytes = None
return super().delete()
def __del__(self):
super().__del__()
if self._verified_bytes:
self.delete()
class BlobFile(AbstractBlob):
"""
A blob existing on the local file system
"""
def __init__(self, loop: asyncio.BaseEventLoop, blob_hash: str, length: typing.Optional[int] = None,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'], asyncio.Task]] = None,
blob_directory: typing.Optional[str] = None):
super().__init__(loop, blob_hash, length, blob_completed_callback, blob_directory)
if not blob_directory or not os.path.isdir(blob_directory):
raise OSError(f"invalid blob directory '{blob_directory}'")
self.file_path = os.path.join(self.blob_directory, self.blob_hash)
if self.file_exists:
file_size = int(os.stat(self.file_path).st_size)
if length and length != file_size:
log.warning("expected %s to be %s bytes, file has %s", self.blob_hash, length, file_size)
self.delete()
else:
self.length = file_size
self.verified.set()
@property
def file_exists(self):
return os.path.isfile(self.file_path)
def is_writeable(self) -> bool:
return super().is_writeable() and not os.path.isfile(self.file_path)
def get_blob_writer(self, peer_address: typing.Optional[str] = None,
peer_port: typing.Optional[str] = None) -> HashBlobWriter:
if self.file_exists:
raise OSError(f"File already exists '{self.file_path}'")
return super().get_blob_writer(peer_address, peer_port)
@contextlib.contextmanager
def _reader_context(self) -> typing.ContextManager[typing.BinaryIO]:
handle = open(self.file_path, 'rb')
try:
yield handle
finally:
handle.close()
def _write_blob(self, blob_bytes: bytes):
with open(self.file_path, 'wb') as f:
f.write(blob_bytes)
def delete(self):
if os.path.isfile(self.file_path):
os.remove(self.file_path)
return super().delete()
@classmethod
async def create_from_unencrypted(
cls, loop: asyncio.BaseEventLoop, blob_dir: typing.Optional[str], key: bytes, iv: bytes,
unencrypted: bytes, blob_num: int,
blob_completed_callback: typing.Optional[typing.Callable[['AbstractBlob'],
asyncio.Task]] = None) -> BlobInfo:
if not blob_dir or not os.path.isdir(blob_dir):
raise OSError(f"cannot create blob in directory: '{blob_dir}'")
return await super().create_from_unencrypted(
loop, blob_dir, key, iv, unencrypted, blob_num, blob_completed_callback
)
-7
View File
@@ -2,13 +2,6 @@ import typing
class BlobInfo:
__slots__ = [
'blob_hash',
'blob_num',
'length',
'iv',
]
def __init__(self, blob_num: int, length: int, iv: str, blob_hash: typing.Optional[str] = None):
self.blob_hash = blob_hash
self.blob_num = blob_num
+22 -60
View File
@@ -2,19 +2,18 @@ import os
import typing
import asyncio
import logging
from lbrynet.blob.blob_file import is_valid_blobhash, BlobFile, BlobBuffer, AbstractBlob
from lbrynet.extras.daemon.storage import SQLiteStorage
from lbrynet.blob.blob_file import BlobFile, is_valid_blobhash
from lbrynet.stream.descriptor import StreamDescriptor
if typing.TYPE_CHECKING:
from lbrynet.conf import Config
from lbrynet.dht.protocol.data_store import DictDataStore
from lbrynet.extras.daemon.storage import SQLiteStorage
log = logging.getLogger(__name__)
class BlobManager:
def __init__(self, loop: asyncio.BaseEventLoop, blob_dir: str, storage: 'SQLiteStorage', config: 'Config',
class BlobFileManager:
def __init__(self, loop: asyncio.BaseEventLoop, blob_dir: str, storage: SQLiteStorage,
node_data_store: typing.Optional['DictDataStore'] = None):
"""
This class stores blobs on the hard disk
@@ -28,59 +27,16 @@ class BlobManager:
self._node_data_store = node_data_store
self.completed_blob_hashes: typing.Set[str] = set() if not self._node_data_store\
else self._node_data_store.completed_blobs
self.blobs: typing.Dict[str, AbstractBlob] = {}
self.config = config
def _get_blob(self, blob_hash: str, length: typing.Optional[int] = None):
if self.config.save_blobs:
return BlobFile(
self.loop, blob_hash, length, self.blob_completed, self.blob_dir
)
else:
if is_valid_blobhash(blob_hash) and os.path.isfile(os.path.join(self.blob_dir, blob_hash)):
return BlobFile(
self.loop, blob_hash, length, self.blob_completed, self.blob_dir
)
return BlobBuffer(
self.loop, blob_hash, length, self.blob_completed, self.blob_dir
)
def get_blob(self, blob_hash, length: typing.Optional[int] = None):
if blob_hash in self.blobs:
if self.config.save_blobs and isinstance(self.blobs[blob_hash], BlobBuffer):
buffer = self.blobs.pop(blob_hash)
if blob_hash in self.completed_blob_hashes:
self.completed_blob_hashes.remove(blob_hash)
self.blobs[blob_hash] = self._get_blob(blob_hash, length)
if buffer.is_readable():
with buffer.reader_context() as reader:
self.blobs[blob_hash].write_blob(reader.read())
if length and self.blobs[blob_hash].length is None:
self.blobs[blob_hash].set_length(length)
else:
self.blobs[blob_hash] = self._get_blob(blob_hash, length)
return self.blobs[blob_hash]
def is_blob_verified(self, blob_hash: str, length: typing.Optional[int] = None) -> bool:
if not is_valid_blobhash(blob_hash):
raise ValueError(blob_hash)
if blob_hash in self.blobs:
return self.blobs[blob_hash].get_is_verified()
if not os.path.isfile(os.path.join(self.blob_dir, blob_hash)):
return False
return self._get_blob(blob_hash, length).get_is_verified()
self.blobs: typing.Dict[str, BlobFile] = {}
async def setup(self) -> bool:
def get_files_in_blob_dir() -> typing.Set[str]:
if not self.blob_dir:
return set()
return {
item.name for item in os.scandir(self.blob_dir) if is_valid_blobhash(item.name)
}
in_blobfiles_dir = await self.loop.run_in_executor(None, get_files_in_blob_dir)
to_add = await self.storage.sync_missing_blobs(in_blobfiles_dir)
if to_add:
self.completed_blob_hashes.update(to_add)
self.completed_blob_hashes.update(await self.storage.sync_missing_blobs(in_blobfiles_dir))
return True
def stop(self):
@@ -89,31 +45,37 @@ class BlobManager:
blob.close()
self.completed_blob_hashes.clear()
def get_blob(self, blob_hash, length: typing.Optional[int] = None):
if blob_hash in self.blobs:
if length and self.blobs[blob_hash].length is None:
self.blobs[blob_hash].set_length(length)
else:
self.blobs[blob_hash] = BlobFile(self.loop, self.blob_dir, blob_hash, length, self.blob_completed)
return self.blobs[blob_hash]
def get_stream_descriptor(self, sd_hash):
return StreamDescriptor.from_stream_descriptor_blob(self.loop, self.blob_dir, self.get_blob(sd_hash))
def blob_completed(self, blob: AbstractBlob) -> asyncio.Task:
async def blob_completed(self, blob: BlobFile):
if blob.blob_hash is None:
raise Exception("Blob hash is None")
if not blob.length:
raise Exception("Blob has a length of 0")
if isinstance(blob, BlobFile):
if blob.blob_hash not in self.completed_blob_hashes:
self.completed_blob_hashes.add(blob.blob_hash)
return self.loop.create_task(self.storage.add_blobs((blob.blob_hash, blob.length), finished=True))
else:
return self.loop.create_task(self.storage.add_blobs((blob.blob_hash, blob.length), finished=False))
if blob.blob_hash not in self.completed_blob_hashes:
self.completed_blob_hashes.add(blob.blob_hash)
await self.storage.add_completed_blob(blob.blob_hash, blob.length)
def check_completed_blobs(self, blob_hashes: typing.List[str]) -> typing.List[str]:
"""Returns of the blobhashes_to_check, which are valid"""
return [blob_hash for blob_hash in blob_hashes if self.is_blob_verified(blob_hash)]
blobs = [self.get_blob(b) for b in blob_hashes]
return [blob.blob_hash for blob in blobs if blob.get_is_verified()]
def delete_blob(self, blob_hash: str):
if not is_valid_blobhash(blob_hash):
raise Exception("invalid blob hash to delete")
if blob_hash not in self.blobs:
if self.blob_dir and os.path.isfile(os.path.join(self.blob_dir, blob_hash)):
if os.path.isfile(os.path.join(self.blob_dir, blob_hash)):
os.remove(os.path.join(self.blob_dir, blob_hash))
else:
self.blobs.pop(blob_hash).delete()
+2 -3
View File
@@ -44,15 +44,16 @@ class HashBlobWriter:
self._hashsum.update(data)
self.len_so_far += len(data)
if self.len_so_far > expected_length:
self.close_handle()
self.finished.set_exception(InvalidDataError(
f'Length so far is greater than the expected length. {self.len_so_far} to {expected_length}'
))
self.close_handle()
return
self.buffer.write(data)
if self.len_so_far == expected_length:
blob_hash = self.calculate_blob_hash()
if blob_hash != self.expected_blob_hash:
self.close_handle()
self.finished.set_exception(InvalidBlobHashError(
f"blob hash is {blob_hash} vs expected {self.expected_blob_hash}"
))
@@ -61,8 +62,6 @@ class HashBlobWriter:
self.close_handle()
def close_handle(self):
if not self.finished.done():
self.finished.cancel()
if self.buffer is not None:
self.buffer.close()
self.buffer = None
+16 -36
View File
@@ -4,9 +4,8 @@ import typing
import binascii
from lbrynet.error import InvalidBlobHashError, InvalidDataError
from lbrynet.blob_exchange.serialization import BlobResponse, BlobRequest
from lbrynet.utils import cache_concurrent
if typing.TYPE_CHECKING:
from lbrynet.blob.blob_file import AbstractBlob
from lbrynet.blob.blob_file import BlobFile
from lbrynet.blob.writer import HashBlobWriter
log = logging.getLogger(__name__)
@@ -18,18 +17,15 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
self.peer_port: typing.Optional[int] = None
self.peer_address: typing.Optional[str] = None
self.peer_timeout = peer_timeout
self.transport: typing.Optional[asyncio.Transport] = None
self.transport: asyncio.Transport = None
self.writer: typing.Optional['HashBlobWriter'] = None
self.blob: typing.Optional['AbstractBlob'] = None
self.writer: 'HashBlobWriter' = None
self.blob: 'BlobFile' = None
self._blob_bytes_received = 0
self._response_fut: typing.Optional[asyncio.Future] = None
self._response_fut: asyncio.Future = None
self.buf = b''
# this is here to handle the race when the downloader is closed right as response_fut gets a result
self.closed = asyncio.Event(loop=self.loop)
def data_received(self, data: bytes):
log.debug("%s:%d -- got %s bytes -- %s bytes on buffer -- %s blob bytes received",
self.peer_address, self.peer_port, len(data), len(self.buf), self._blob_bytes_received)
@@ -39,9 +35,6 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
return
if not self._response_fut:
log.warning("Protocol received data before expected, probable race on keep alive. Closing transport.")
return self.close()
if self._blob_bytes_received and not self.writer.closed():
return self._write(data)
@@ -70,7 +63,8 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
# write blob bytes if we're writing a blob and have blob bytes to write
self._write(response.blob_data)
def _write(self, data: bytes):
def _write(self, data):
if len(data) > (self.blob.get_length() - self._blob_bytes_received):
data = data[:(self.blob.get_length() - self._blob_bytes_received)]
log.warning("got more than asked from %s:%d, probable sendfile bug", self.peer_address, self.peer_port)
@@ -93,7 +87,6 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
:return: download success (bool), keep connection (bool)
"""
request = BlobRequest.make_request_for_blob_hash(self.blob.blob_hash)
blob_hash = self.blob.blob_hash
try:
msg = request.serialize()
log.debug("send request to %s:%i -> %s", self.peer_address, self.peer_port, msg.decode())
@@ -102,10 +95,6 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
availability_response = response.get_availability_response()
price_response = response.get_price_response()
blob_response = response.get_blob_response()
if self.closed.is_set():
msg = f"cancelled blob request for {blob_hash} immediately after we got a response"
log.warning(msg)
raise asyncio.CancelledError(msg)
if (not blob_response or blob_response.error) and\
(not availability_response or not availability_response.available_blobs):
log.warning("%s not in availability response from %s:%i", self.blob.blob_hash, self.peer_address,
@@ -144,7 +133,6 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
return self._blob_bytes_received, self.close()
def close(self):
self.closed.set()
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
if self.writer and not self.writer.closed():
@@ -157,21 +145,18 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
self.transport = None
self.buf = b''
async def download_blob(self, blob: 'AbstractBlob') -> typing.Tuple[int, typing.Optional[asyncio.Transport]]:
self.closed.clear()
blob_hash = blob.blob_hash
if blob.get_is_verified() or not blob.is_writeable():
async def download_blob(self, blob: 'BlobFile') -> typing.Tuple[int, typing.Optional[asyncio.Transport]]:
if blob.get_is_verified() or blob.file_exists or blob.blob_write_lock.locked():
return 0, self.transport
try:
self._blob_bytes_received = 0
self.blob, self.writer = blob, blob.get_blob_writer(self.peer_address, self.peer_port)
self.blob, self.writer, self._blob_bytes_received = blob, blob.open_for_writing(), 0
self._response_fut = asyncio.Future(loop=self.loop)
return await self._download_blob()
except OSError as e:
log.error("race happened downloading from %s:%i", self.peer_address, self.peer_port)
# i'm not sure how to fix this race condition - jack
log.warning("race happened downloading %s from %s:%i", blob_hash, self.peer_address, self.peer_port)
# return self._blob_bytes_received, self.transport
raise
log.exception(e)
return self._blob_bytes_received, self.transport
except asyncio.TimeoutError:
if self._response_fut and not self._response_fut.done():
self._response_fut.cancel()
@@ -180,10 +165,6 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
except asyncio.CancelledError:
self.close()
raise
finally:
if self.writer and not self.writer.closed():
self.writer.close_handle()
self.writer = None
def connection_made(self, transport: asyncio.Transport):
self.transport = transport
@@ -196,10 +177,9 @@ class BlobExchangeClientProtocol(asyncio.Protocol):
self.close()
@cache_concurrent
async def request_blob(loop: asyncio.BaseEventLoop, blob: 'AbstractBlob', address: str, tcp_port: int,
async def request_blob(loop: asyncio.BaseEventLoop, blob: 'BlobFile', address: str, tcp_port: int,
peer_connect_timeout: float, blob_download_timeout: float,
connected_transport: asyncio.Transport = None, connection_id: int = 0)\
connected_transport: asyncio.Transport = None)\
-> typing.Tuple[int, typing.Optional[asyncio.Transport]]:
"""
Returns [<downloaded blob>, <keep connection>]
@@ -216,7 +196,7 @@ async def request_blob(loop: asyncio.BaseEventLoop, blob: 'AbstractBlob', addres
if not connected_transport:
await asyncio.wait_for(loop.create_connection(lambda: protocol, address, tcp_port),
peer_connect_timeout, loop=loop)
if blob.get_is_verified() or not blob.is_writeable():
if blob.get_is_verified() or blob.file_exists:
# file exists but not verified means someone is writing right now, give it time, come back later
return 0, connected_transport
return await protocol.download_blob(blob)
+56 -41
View File
@@ -1,23 +1,21 @@
import asyncio
import typing
import logging
from lbrynet.utils import cache_concurrent
from lbrynet.utils import drain_tasks
from lbrynet.blob_exchange.client import request_blob
if typing.TYPE_CHECKING:
from lbrynet.conf import Config
from lbrynet.dht.node import Node
from lbrynet.dht.peer import KademliaPeer
from lbrynet.blob.blob_manager import BlobManager
from lbrynet.blob.blob_file import AbstractBlob
from lbrynet.blob.blob_manager import BlobFileManager
from lbrynet.blob.blob_file import BlobFile
log = logging.getLogger(__name__)
class BlobDownloader:
BAN_FACTOR = 2.0 # fixme: when connection manager gets implemented, move it out from here
BAN_CAP = 30.0*60.0 # maximum time a node remains banned before being retried
def __init__(self, loop: asyncio.BaseEventLoop, config: 'Config', blob_manager: 'BlobManager',
BAN_TIME = 10.0 # fixme: when connection manager gets implemented, move it out from here
def __init__(self, loop: asyncio.BaseEventLoop, config: 'Config', blob_manager: 'BlobFileManager',
peer_queue: asyncio.Queue):
self.loop = loop
self.config = config
@@ -26,16 +24,21 @@ class BlobDownloader:
self.active_connections: typing.Dict['KademliaPeer', asyncio.Task] = {} # active request_blob calls
self.ignored: typing.Dict['KademliaPeer', int] = {}
self.scores: typing.Dict['KademliaPeer', int] = {}
self.failures: typing.Dict['KademliaPeer', int] = {}
self.connections: typing.Dict['KademliaPeer', asyncio.Transport] = {}
self.is_running = asyncio.Event(loop=self.loop)
self.time_since_last_blob = loop.time()
def should_race_continue(self, blob: 'AbstractBlob'):
def should_race_continue(self, blob: 'BlobFile'):
if len(self.active_connections) >= self.config.max_connections_per_download:
return False
return not (blob.get_is_verified() or not blob.is_writeable())
# if a peer won 3 or more blob races and is active as a downloader, stop the race so bandwidth improves
# the safe net side is that any failure will reset the peer score, triggering the race back
# TODO: this is a good idea for low bandwidth, but doesnt play nice on high bandwidth
# for peer, task in self.active_connections.items():
# if self.scores.get(peer, 0) >= 0 and self.rounds_won.get(peer, 0) >= 3 and not task.done():
# return False
return not (blob.get_is_verified() or blob.file_exists)
async def request_blob_from_peer(self, blob: 'AbstractBlob', peer: 'KademliaPeer', connection_id: int = 0):
async def request_blob_from_peer(self, blob: 'BlobFile', peer: 'KademliaPeer'):
if blob.get_is_verified():
return
self.scores[peer] = self.scores.get(peer, 0) - 1 # starts losing score, to account for cancelled ones
@@ -43,80 +46,92 @@ class BlobDownloader:
start = self.loop.time()
bytes_received, transport = await request_blob(
self.loop, blob, peer.address, peer.tcp_port, self.config.peer_connect_timeout,
self.config.blob_download_timeout, connected_transport=transport, connection_id=connection_id
self.config.blob_download_timeout, connected_transport=transport
)
if bytes_received == blob.get_length():
self.time_since_last_blob = self.loop.time()
if not transport and peer not in self.ignored:
self.ignored[peer] = self.loop.time()
log.debug("drop peer %s:%i", peer.address, peer.tcp_port)
self.failures[peer] = self.failures.get(peer, 0) + 1
if peer in self.connections:
del self.connections[peer]
elif transport:
log.debug("keep peer %s:%i", peer.address, peer.tcp_port)
self.failures[peer] = 0
self.connections[peer] = transport
elapsed = self.loop.time() - start
self.scores[peer] = bytes_received / elapsed if bytes_received and elapsed else 0
rough_speed = (bytes_received / (self.loop.time() - start)) if bytes_received else 0
self.scores[peer] = rough_speed
async def new_peer_or_finished(self):
active_tasks = list(self.active_connections.values()) + [asyncio.sleep(1)]
await asyncio.wait(active_tasks, loop=self.loop, return_when='FIRST_COMPLETED')
async def new_peer_or_finished(self, blob: 'BlobFile'):
async def get_and_re_add_peers():
try:
new_peers = await asyncio.wait_for(self.peer_queue.get(), timeout=1.0)
self.peer_queue.put_nowait(new_peers)
except asyncio.TimeoutError:
pass
tasks = [self.loop.create_task(get_and_re_add_peers()), self.loop.create_task(blob.verified.wait())]
active_tasks = list(self.active_connections.values())
try:
await asyncio.wait(tasks + active_tasks, loop=self.loop, return_when='FIRST_COMPLETED')
finally:
drain_tasks(tasks)
def cleanup_active(self):
to_remove = [peer for (peer, task) in self.active_connections.items() if task.done()]
for peer in to_remove:
del self.active_connections[peer]
self.clearbanned()
def clearbanned(self):
now = self.loop.time()
self.ignored = dict((
(peer, when) for (peer, when) in self.ignored.items()
if (now - when) < min(self.BAN_CAP, (self.failures.get(peer, 0) ** self.BAN_FACTOR))
))
if now - self.time_since_last_blob > 60.0:
return
forgiven = [banned_peer for banned_peer, when in self.ignored.items() if now - when > self.BAN_TIME]
self.peer_queue.put_nowait(forgiven)
for banned_peer in forgiven:
self.ignored.pop(banned_peer)
@cache_concurrent
async def download_blob(self, blob_hash: str, length: typing.Optional[int] = None,
connection_id: int = 0) -> 'AbstractBlob':
async def download_blob(self, blob_hash: str, length: typing.Optional[int] = None) -> 'BlobFile':
blob = self.blob_manager.get_blob(blob_hash, length)
if blob.get_is_verified():
return blob
self.is_running.set()
try:
while not blob.get_is_verified() and self.is_running.is_set():
batch: typing.Set['KademliaPeer'] = set()
while not blob.get_is_verified():
batch: typing.List['KademliaPeer'] = []
while not self.peer_queue.empty():
batch.update(self.peer_queue.get_nowait())
if batch:
self.peer_queue.put_nowait(list(batch))
batch.extend(self.peer_queue.get_nowait())
batch.sort(key=lambda peer: self.scores.get(peer, 0), reverse=True)
log.debug(
"running, %d peers, %d ignored, %d active",
len(batch), len(self.ignored), len(self.active_connections)
)
for peer in sorted(batch, key=lambda peer: self.scores.get(peer, 0), reverse=True):
for peer in batch:
if not self.should_race_continue(blob):
break
if peer not in self.active_connections and peer not in self.ignored:
log.debug("request %s from %s:%i", blob_hash[:8], peer.address, peer.tcp_port)
t = self.loop.create_task(self.request_blob_from_peer(blob, peer, connection_id))
t = self.loop.create_task(self.request_blob_from_peer(blob, peer))
self.active_connections[peer] = t
await self.new_peer_or_finished()
await self.new_peer_or_finished(blob)
self.cleanup_active()
if batch:
self.peer_queue.put_nowait(set(batch).difference(self.ignored))
else:
self.clearbanned()
blob.close()
log.debug("downloaded %s", blob_hash[:8])
return blob
finally:
blob.close()
while self.active_connections:
self.active_connections.popitem()[1].cancel()
def close(self):
self.scores.clear()
self.ignored.clear()
self.is_running.clear()
for transport in self.connections.values():
transport.close()
async def download_blob(loop, config: 'Config', blob_manager: 'BlobManager', node: 'Node',
blob_hash: str) -> 'AbstractBlob':
async def download_blob(loop, config: 'Config', blob_manager: 'BlobFileManager', node: 'Node',
blob_hash: str) -> 'BlobFile':
search_queue = asyncio.Queue(loop=loop, maxsize=config.max_connections_per_download)
search_queue.put_nowait(blob_hash)
peer_queue, accumulate_task = node.accumulate_peers(search_queue)
+3 -3
View File
@@ -8,13 +8,13 @@ from lbrynet.blob_exchange.serialization import BlobAvailabilityResponse, BlobPr
BlobPaymentAddressResponse
if typing.TYPE_CHECKING:
from lbrynet.blob.blob_manager import BlobManager
from lbrynet.blob.blob_manager import BlobFileManager
log = logging.getLogger(__name__)
class BlobServerProtocol(asyncio.Protocol):
def __init__(self, loop: asyncio.BaseEventLoop, blob_manager: 'BlobManager', lbrycrd_address: str):
def __init__(self, loop: asyncio.BaseEventLoop, blob_manager: 'BlobFileManager', lbrycrd_address: str):
self.loop = loop
self.blob_manager = blob_manager
self.server_task: asyncio.Task = None
@@ -94,7 +94,7 @@ class BlobServerProtocol(asyncio.Protocol):
class BlobServer:
def __init__(self, loop: asyncio.BaseEventLoop, blob_manager: 'BlobManager', lbrycrd_address: str):
def __init__(self, loop: asyncio.BaseEventLoop, blob_manager: 'BlobFileManager', lbrycrd_address: str):
self.loop = loop
self.blob_manager = blob_manager
self.server_task: asyncio.Task = None
+6 -8
View File
@@ -21,6 +21,10 @@ CURRENCIES = {
'LBC': {'type': 'crypto'},
'USD': {'type': 'fiat'},
}
SLACK_WEBHOOK = (
'nUE0pUZ6Yl9bo29epl5moTSwnl5wo20ip2IlqzywMKZiIQSFZR5'
'AHx4mY0VmF0WQZ1ESEP9kMHZlp1WzJwWOoKN3ImR1M2yUAaMyqGZ='
)
HEADERS_FILE_SHA256_CHECKSUM = (
366295, 'b0c8197153a33ccbc52fb81a279588b6015b68b7726f73f6a2b81f7e25bfe4b9'
)
@@ -480,12 +484,10 @@ class Config(CLIConfig):
# protocol timeouts
download_timeout = Float("Cumulative timeout for a stream to begin downloading before giving up", 30.0)
blob_download_timeout = Float("Timeout to download a blob from a peer", 30.0)
peer_connect_timeout = Float("Timeout to establish a TCP connection to a peer", 3.0)
peer_connect_timeout = Float("Timeout to establish a TCP connection to a peer", 2.0)
node_rpc_timeout = Float("Timeout when making a DHT request", constants.rpc_timeout)
# blob announcement and download
save_blobs = Toggle("Save encrypted blob files for hosting, otherwise download blobs to memory only.", True)
announce_head_and_sd_only = Toggle(
"Announce only the descriptor and first (rather than all) data blob for a stream to the DHT", True,
previous_names=['announce_head_blobs_only']
@@ -495,7 +497,7 @@ class Config(CLIConfig):
previous_names=['concurrent_announcers']
)
max_connections_per_download = Integer(
"Maximum number of peers to connect to while downloading a blob", 4,
"Maximum number of peers to connect to while downloading a blob", 8,
previous_names=['max_connections_per_stream']
)
fixed_peer_delay = Float(
@@ -530,16 +532,12 @@ class Config(CLIConfig):
('lbrynet4.lbry.io', 4444) # ASIA
])
# comment server uses a single string because requests are made to the /api resource
comment_server = String("Server to store user-generated metadata.", "http://comments.lbry.com:2903/api")
# blockchain
blockchain_name = String("Blockchain name - lbrycrd_main, lbrycrd_regtest, or lbrycrd_testnet", 'lbrycrd_main')
s3_headers_depth = Integer("download headers from s3 when the local height is more than 10 chunks behind", 96 * 10)
cache_time = Integer("Time to cache resolved claims", 150) # TODO: use this
# daemon
save_files = Toggle("Save downloaded files when calling `get` by default", True)
components_to_skip = Strings("components which will be skipped during start-up of daemon", [])
share_usage_data = Toggle(
"Whether to share usage stats and diagnostic info with LBRY.", True,
+44 -33
View File
@@ -13,44 +13,55 @@ class BlobAnnouncer:
self.loop = loop
self.node = node
self.storage = storage
self.pending_call: asyncio.Handle = None
self.announce_task: asyncio.Task = None
self.running = False
self.announce_queue: typing.List[str] = []
async def _submit_announcement(self, blob_hash):
try:
peers = len(await self.node.announce_blob(blob_hash))
if peers > 4:
return blob_hash
else:
log.warning("failed to announce %s, could only find %d peers, retrying soon.", blob_hash[:8], peers)
except Exception as err:
if isinstance(err, asyncio.CancelledError):
raise err
log.warning("error announcing %s: %s", blob_hash[:8], str(err))
async def _announce(self, batch_size: typing.Optional[int] = 10):
while batch_size:
if not self.node.joined.is_set():
await self.node.joined.wait()
self.announce_queue.extend(await self.storage.get_blobs_to_announce())
log.debug("announcer task wake up, %d blobs to announce", len(self.announce_queue))
while len(self.announce_queue):
log.info("%i blobs to announce", len(self.announce_queue))
announced = await asyncio.gather(*[
self._submit_announcement(
self.announce_queue.pop()) for _ in range(batch_size) if self.announce_queue
], loop=self.loop)
announced = list(filter(None, announced))
if announced:
await self.storage.update_last_announced_blobs(announced)
log.info("announced %i blobs", len(announced))
await asyncio.sleep(60)
if not batch_size:
return
if not self.node.joined.is_set():
await self.node.joined.wait()
blob_hashes = await self.storage.get_blobs_to_announce()
if blob_hashes:
self.announce_queue.extend(blob_hashes)
log.info("%i blobs to announce", len(blob_hashes))
batch = []
while len(self.announce_queue):
cnt = 0
announced = []
while self.announce_queue and cnt < batch_size:
blob_hash = self.announce_queue.pop()
announced.append(blob_hash)
batch.append(self.node.announce_blob(blob_hash))
cnt += 1
to_await = []
while batch:
to_await.append(batch.pop())
if to_await:
await asyncio.gather(*tuple(to_await), loop=self.loop)
await self.storage.update_last_announced_blobs(announced)
log.info("announced %i blobs", len(announced))
if self.running:
self.pending_call = self.loop.call_later(60, self.announce, batch_size)
def start(self, batch_size: typing.Optional[int] = 10):
assert not self.announce_task or self.announce_task.done(), "already running"
def announce(self, batch_size: typing.Optional[int] = 10):
self.announce_task = self.loop.create_task(self._announce(batch_size))
def start(self, batch_size: typing.Optional[int] = 10):
if self.running:
raise Exception("already running")
self.running = True
self.announce(batch_size)
def stop(self):
if self.announce_task and not self.announce_task.done():
self.announce_task.cancel()
self.running = False
if self.pending_call:
if not self.pending_call.cancelled():
self.pending_call.cancel()
self.pending_call = None
if self.announce_task:
if not (self.announce_task.done() or self.announce_task.cancelled()):
self.announce_task.cancel()
self.announce_task = None
+1
View File
@@ -201,6 +201,7 @@ class Node:
try:
async with peer_generator as junction:
yield junction
await peer_generator.finished.wait()
finally:
if add_hashes_task and not (add_hashes_task.done() or add_hashes_task.cancelled()):
add_hashes_task.cancel()
-9
View File
@@ -135,15 +135,6 @@ class PeerManager:
class KademliaPeer:
__slots__ = [
'loop',
'_node_id',
'address',
'udp_port',
'tcp_port',
'protocol_version',
]
def __init__(self, loop: asyncio.BaseEventLoop, address: str, node_id: typing.Optional[bytes] = None,
udp_port: typing.Optional[int] = None, tcp_port: typing.Optional[int] = None):
if node_id is not None:
@@ -26,11 +26,12 @@ class AsyncGeneratorJunction:
def __init__(self, loop: asyncio.BaseEventLoop, queue: typing.Optional[asyncio.Queue] = None):
self.loop = loop
self.__iterator_queue = asyncio.Queue(loop=loop)
self.result_queue = queue or asyncio.Queue(loop=loop)
self.tasks: typing.List[asyncio.Task] = []
self.running_iterators: typing.Dict[typing.AsyncGenerator, bool] = {}
self.generator_queue: asyncio.Queue = asyncio.Queue(loop=self.loop)
self.can_iterate = asyncio.Event(loop=self.loop)
self.finished = asyncio.Event(loop=self.loop)
@property
def running(self):
@@ -41,16 +42,15 @@ class AsyncGeneratorJunction:
try:
async for item in iterator:
self.result_queue.put_nowait(item)
self.__iterator_queue.put_nowait(item)
finally:
self.running_iterators[iterator] = False
if not self.running:
self.__iterator_queue.put_nowait(StopAsyncIteration)
while True:
async_gen: typing.Union[typing.AsyncGenerator, 'AsyncGeneratorType'] = await self.generator_queue.get()
self.running_iterators[async_gen] = True
self.tasks.append(self.loop.create_task(iterate(async_gen)))
if not self.can_iterate.is_set():
self.can_iterate.set()
def add_generator(self, async_gen: typing.Union[typing.AsyncGenerator, 'AsyncGeneratorType']):
"""
@@ -62,10 +62,14 @@ class AsyncGeneratorJunction:
return self
async def __anext__(self):
result = await self.__iterator_queue.get()
if result is StopAsyncIteration:
raise result
return result
if not self.can_iterate.is_set():
await self.can_iterate.wait()
if not self.running:
raise StopAsyncIteration()
try:
return await self.result_queue.get()
finally:
self.awaiting = None
def aclose(self):
async def _aclose():
@@ -76,6 +80,8 @@ class AsyncGeneratorJunction:
self.running_iterators[iterator] = False
drain_tasks(self.tasks)
raise StopAsyncIteration()
if not self.finished.is_set():
self.finished.set()
return self.loop.create_task(_aclose())
async def __aenter__(self):
+2 -4
View File
@@ -5,7 +5,7 @@ from itertools import chain
import typing
import logging
from lbrynet.dht import constants
from lbrynet.dht.error import RemoteException, TransportNotConnected
from lbrynet.dht.error import RemoteException
from lbrynet.dht.protocol.distance import Distance
from typing import TYPE_CHECKING
@@ -169,8 +169,6 @@ class IterativeFinder:
log.warning(str(err))
self.active.discard(peer)
return
except TransportNotConnected:
return self.aclose()
except RemoteException:
return
return await self._handle_probe_result(peer, response)
@@ -217,7 +215,7 @@ class IterativeFinder:
await self._search_round()
if self.running:
self.delayed_calls.append(self.loop.call_later(delay, self._search))
except (asyncio.CancelledError, StopAsyncIteration, TransportNotConnected):
except (asyncio.CancelledError, StopAsyncIteration):
if self.running:
self.loop.call_soon(self.aclose)
+1 -5
View File
@@ -524,10 +524,6 @@ class KademliaProtocol(DatagramProtocol):
response = await asyncio.wait_for(response_fut, self.rpc_timeout)
self.peer_manager.report_last_replied(peer.address, peer.udp_port)
return response
except asyncio.CancelledError:
if not response_fut.done():
response_fut.cancel()
raise
except (asyncio.TimeoutError, RemoteException):
self.peer_manager.report_failure(peer.address, peer.udp_port)
if self.peer_manager.peer_is_good(peer) is False:
@@ -544,7 +540,7 @@ class KademliaProtocol(DatagramProtocol):
async def _send(self, peer: 'KademliaPeer', message: typing.Union[RequestDatagram, ResponseDatagram,
ErrorDatagram]):
if not self.transport or self.transport.is_closing():
if not self.transport:
raise TransportNotConnected()
data = message.bencode()
-6
View File
@@ -32,12 +32,6 @@ class DownloadDataTimeout(Exception):
self.download = download
class ResolveTimeout(Exception):
def __init__(self, uri):
super().__init__(f'Failed to resolve "{uri}" within the timeout')
self.uri = uri
class RequestCanceledError(Exception):
pass
+6 -3
View File
@@ -25,7 +25,7 @@ def display(data):
print(json.dumps(data, indent=2))
async def execute_command(conf, method, params, callback=display):
async def execute_command(conf, method, params):
async with aiohttp.ClientSession() as session:
try:
message = {'method': method, 'params': params}
@@ -33,9 +33,12 @@ async def execute_command(conf, method, params, callback=display):
try:
data = await resp.json()
if 'result' in data:
return callback(data['result'])
display(data['result'])
elif 'error' in data:
return callback(data['error'])
if 'message' in data['error']:
display(data['error']['message'])
else:
display(data['error'])
except Exception as e:
log.exception('Could not process response from server:', exc_info=e)
except aiohttp.ClientConnectionError:
+19 -17
View File
@@ -12,18 +12,19 @@ from aioupnp import __version__ as aioupnp_version
from aioupnp.upnp import UPnP
from aioupnp.fault import UPnPError
import lbrynet.schema
from lbrynet import utils
from lbrynet.conf import HEADERS_FILE_SHA256_CHECKSUM
from lbrynet.dht.node import Node
from lbrynet.dht.blob_announcer import BlobAnnouncer
from lbrynet.blob.blob_manager import BlobManager
from lbrynet.blob.blob_manager import BlobFileManager
from lbrynet.blob_exchange.server import BlobServer
from lbrynet.stream.stream_manager import StreamManager
from lbrynet.extras.daemon.Component import Component
from lbrynet.extras.daemon.exchange_rate_manager import ExchangeRateManager
from lbrynet.extras.daemon.storage import SQLiteStorage
from lbrynet.wallet import LbryWalletManager
from lbrynet.wallet import Network
from lbrynet.extras.wallet import LbryWalletManager
from lbrynet.extras.wallet import Network
log = logging.getLogger(__name__)
@@ -63,7 +64,7 @@ class DatabaseComponent(Component):
@staticmethod
def get_current_db_revision():
return 11
return 9
@property
def revision_filename(self):
@@ -251,20 +252,23 @@ class WalletComponent(Component):
async def get_status(self):
if self.wallet_manager and self.running:
local_height = self.wallet_manager.ledger.headers.height
remote_height = await self.wallet_manager.ledger.network.get_server_height()
local_height = self.wallet_manager.network.get_local_height()
remote_height = self.wallet_manager.network.get_server_height()
best_hash = self.wallet_manager.get_best_blockhash()
return {
'blocks': max(local_height, 0),
'blocks_behind': max(remote_height - local_height, 0),
'best_blockhash': best_hash,
'is_encrypted': self.wallet_manager.use_encryption,
'is_encrypted': self.wallet_manager.wallet.use_encryption,
'is_locked': not self.wallet_manager.is_wallet_unlocked,
}
async def start(self):
log.info("Starting torba wallet")
self.wallet_manager = await LbryWalletManager.from_lbrynet_config(self.conf)
storage = self.component_manager.get_component(DATABASE_COMPONENT)
lbrynet.schema.BLOCKCHAIN_NAME = self.conf.blockchain_name
self.wallet_manager = await LbryWalletManager.from_lbrynet_config(self.conf, storage)
self.wallet_manager.old_db = storage
await self.wallet_manager.start()
async def stop(self):
@@ -278,10 +282,10 @@ class BlobComponent(Component):
def __init__(self, component_manager):
super().__init__(component_manager)
self.blob_manager: BlobManager = None
self.blob_manager: BlobFileManager = None
@property
def component(self) -> typing.Optional[BlobManager]:
def component(self) -> typing.Optional[BlobFileManager]:
return self.blob_manager
async def start(self):
@@ -291,10 +295,8 @@ class BlobComponent(Component):
dht_node: Node = self.component_manager.get_component(DHT_COMPONENT)
if dht_node:
data_store = dht_node.protocol.data_store
blob_dir = os.path.join(self.conf.data_dir, 'blobfiles')
if not os.path.isdir(blob_dir):
os.mkdir(blob_dir)
self.blob_manager = BlobManager(asyncio.get_event_loop(), blob_dir, storage, self.conf, data_store)
self.blob_manager = BlobFileManager(asyncio.get_event_loop(), os.path.join(self.conf.data_dir, "blobfiles"),
storage, data_store)
return await self.blob_manager.setup()
async def stop(self):
@@ -453,7 +455,7 @@ class PeerProtocolServerComponent(Component):
async def start(self):
log.info("start blob server")
upnp = self.component_manager.get_component(UPNP_COMPONENT)
blob_manager: BlobManager = self.component_manager.get_component(BLOB_COMPONENT)
blob_manager: BlobFileManager = self.component_manager.get_component(BLOB_COMPONENT)
wallet: LbryWalletManager = self.component_manager.get_component(WALLET_COMPONENT)
peer_port = self.conf.tcp_port
address = await wallet.get_unused_address()
@@ -487,7 +489,7 @@ class UPnPComponent(Component):
while True:
if now:
await self._maintain_redirects()
await asyncio.sleep(360, loop=self.component_manager.loop)
await asyncio.sleep(360)
async def _maintain_redirects(self):
# setup the gateway if necessary
@@ -530,7 +532,7 @@ class UPnPComponent(Component):
self.upnp_redirects.update(upnp_redirects)
except (asyncio.TimeoutError, UPnPError):
self.upnp = None
return
return self._maintain_redirects()
elif self.upnp: # check existing redirects are still active
found = set()
mappings = await self.upnp.get_redirects()
File diff suppressed because it is too large. Load diff
+24 -3
View File
@@ -1,6 +1,27 @@
import aiohttp
import logging
from lbrynet.conf import Config
from lbrynet.extras.cli import execute_command
log = logging.getLogger(__name__)
USER_AGENT = "AuthServiceProxy/0.1"
TWISTED_SECURE_SESSION = "TWISTED_SECURE_SESSION"
TWISTED_SESSION = "TWISTED_SESSION"
LBRY_SECRET = "LBRY_SECRET"
HTTP_TIMEOUT = 30
def daemon_rpc(conf: Config, method: str, **kwargs):
return execute_command(conf, method, kwargs, callback=lambda data: data)
class JSONRPCException(Exception):
def __init__(self, rpc_error):
super().__init__()
self.error = rpc_error
async def daemon_rpc(conf: Config, method: str, *args, **kwargs):
async with aiohttp.ClientSession() as session:
message = {'method': method, 'params': [args, kwargs]}
async with session.get(conf.api_connection_url, json=message) as resp:
data = await resp.json()
if 'result' in data:
return data['result']
elif 'error' in data:
raise JSONRPCException(data['error'])
-31
View File
@@ -1,31 +0,0 @@
import logging
import aiohttp
log = logging.getLogger(__name__)
def rpc_body(method: str, rpc_id: any, **params) -> dict:
return {'jsonrpc': '2.0', 'id': rpc_id, 'method': method, 'params': {**params}}
async def jsonrpc_post(url: str, method: str, **params) -> any:
clean = params.pop('clean', True)
response = (await jsonrpc_batch(url, [rpc_body(method, 1, **params)]))[0]
if clean:
if 'error' in response:
return response['error']
return response['result']
else:
return response
async def jsonrpc_batch(url: str, calls: list, batch_size: int = 50, clean: bool = False) -> list:
headers = {'Content-Type': 'application/json'}
complete = []
batch_size = max(batch_size, 50)
for i in range(0, len(calls), batch_size):
async with aiohttp.request('POST', url, headers=headers, json=calls[i:i+batch_size]) as response:
complete += await response.json()
if clean:
complete = [body['result'] if 'result' in body else None for body in complete]
return complete
@@ -2,7 +2,6 @@ import asyncio
import time
import logging
import json
from decimal import Decimal
from lbrynet.error import InvalidExchangeRateResponse, CurrencyConversionError
from lbrynet.utils import aiohttp_request
@@ -227,12 +226,12 @@ class ExchangeRateManager:
for market in self.market_feeds:
if (market.rate_is_initialized() and market.is_online() and
market.rate.currency_pair == (from_currency, to_currency)):
return amount * Decimal(market.rate.spot)
return amount * market.rate.spot
for market in self.market_feeds:
if (market.rate_is_initialized() and market.is_online() and
market.rate.currency_pair[0] == from_currency):
return self.convert_currency(
market.rate.currency_pair[1], to_currency, amount * Decimal(market.rate.spot))
market.rate.currency_pair[1], to_currency, amount * market.rate.spot)
raise CurrencyConversionError(
f'Unable to convert {amount} from {from_currency} to {to_currency}')
+30 -148
View File
@@ -4,117 +4,25 @@ from binascii import hexlify
from datetime import datetime
from json import JSONEncoder
from ecdsa import BadSignatureError
from lbrynet.schema.claim import Claim
from lbrynet.wallet.ledger import MainNetLedger, Account
from lbrynet.wallet.transaction import Transaction, Output
from lbrynet.wallet.dewies import dewies_to_lbc
from lbrynet.stream.managed_stream import ManagedStream
from lbrynet.extras.wallet import MainNetLedger
from lbrynet.extras.wallet.transaction import Transaction, Output
from lbrynet.extras.wallet.dewies import dewies_to_lbc
log = logging.getLogger(__name__)
def encode_txo_doc():
return {
'txid': "hash of transaction in hex",
'nout': "position in the transaction",
'height': "block where transaction was recorded",
'amount': "value of the txo as a decimal",
'address': "address of who can spend the txo",
'confirmations': "number of confirmed blocks",
'is_change': "payment to change address, only available when it can be determined",
'is_mine': "payment to one of your accounts, only available when it can be determined",
'type': "one of 'claim', 'support' or 'payment'",
'name': "when type is 'claim' or 'support', this is the claim name",
'claim_id': "when type is 'claim' or 'support', this is the claim id",
'claim_op': "when type is 'claim', this determines if it is 'create' or 'update'",
'value': "when type is 'claim' or 'support' with payload, this is the decoded protobuf payload",
'value_type': "determines the type of the 'value' field: 'channel', 'stream', etc",
'protobuf': "hex encoded raw protobuf version of 'value' field",
'permanent_url': "when type is 'claim' or 'support', this is the long permanent claim URL",
'signing_channel': "for signed claims only, metadata of signing channel",
'is_channel_signature_valid': "for signed claims only, whether signature is valid",
}
def encode_tx_doc():
return {
'txid': "hash of transaction in hex",
'height': "block where transaction was recorded",
'inputs': [encode_txo_doc()],
'outputs': [encode_txo_doc()],
'total_input': "sum of inputs as a decimal",
'total_output': "sum of outputs, sans fee, as a decimal",
'total_fee': "fee amount",
'hex': "entire transaction encoded in hex",
}
def encode_account_doc():
return {
'id': 'account_id',
'is_default': 'this account is used by default',
'ledger': 'name of crypto currency and network',
'name': 'optional account name',
'seed': 'human friendly words from which account can be recreated',
'encrypted': 'if account is encrypted',
'private_key': 'extended private key',
'public_key': 'extended public key',
'address_generator': 'settings for generating addresses',
'modified_on': 'date of last modification to account settings'
}
def encode_file_doc():
return {
'completed': '(bool) true if download is completed',
'file_name': '(str) name of file',
'download_directory': '(str) download directory',
'points_paid': '(float) credit paid to download file',
'stopped': '(bool) true if download is stopped',
'stream_hash': '(str) stream hash of file',
'stream_name': '(str) stream name',
'suggested_file_name': '(str) suggested file name',
'sd_hash': '(str) sd hash of file',
'download_path': '(str) download path of file',
'mime_type': '(str) mime type of file',
'key': '(str) key attached to file',
'total_bytes_lower_bound': '(int) lower bound file size in bytes',
'total_bytes': '(int) file upper bound size in bytes',
'written_bytes': '(int) written size in bytes',
'blobs_completed': '(int) number of fully downloaded blobs',
'blobs_in_stream': '(int) total blobs on stream',
'blobs_remaining': '(int) total blobs remaining to download',
'status': '(str) downloader status',
'claim_id': '(str) None if claim is not found else the claim id',
'txid': '(str) None if claim is not found else the transaction id',
'nout': '(int) None if claim is not found else the transaction output index',
'outpoint': '(str) None if claim is not found else the tx and output',
'metadata': '(dict) None if claim is not found else the claim metadata',
'channel_claim_id': '(str) None if claim is not found or not signed',
'channel_name': '(str) None if claim is not found or not signed',
'claim_name': '(str) None if claim is not found else the claim name'
}
class JSONResponseEncoder(JSONEncoder):
def __init__(self, *args, ledger: MainNetLedger, include_protobuf=False, **kwargs):
def __init__(self, *args, ledger: MainNetLedger, **kwargs):
super().__init__(*args, **kwargs)
self.ledger = ledger
self.include_protobuf = include_protobuf
def default(self, obj): # pylint: disable=method-hidden
if isinstance(obj, Account):
return self.encode_account(obj)
if isinstance(obj, ManagedStream):
return self.encode_file(obj)
if isinstance(obj, Transaction):
return self.encode_transaction(obj)
if isinstance(obj, Output):
return self.encode_output(obj)
if isinstance(obj, Claim):
return self.encode_claim(obj)
if isinstance(obj, datetime):
return obj.strftime("%Y%m%dT%H:%M:%S")
if isinstance(obj, Decimal):
@@ -135,88 +43,62 @@ class JSONResponseEncoder(JSONEncoder):
'hex': hexlify(tx.raw).decode(),
}
def encode_output(self, txo, check_signature=True):
def encode_output(self, txo):
tx_height = txo.tx_ref.height
best_height = self.ledger.headers.height
output = {
'txid': txo.tx_ref.id,
'nout': txo.position,
'height': tx_height,
'amount': dewies_to_lbc(txo.amount),
'address': txo.get_address(self.ledger),
'confirmations': (best_height+1) - tx_height if tx_height > 0 else tx_height,
'timestamp': self.ledger.headers[tx_height]['timestamp'] if tx_height > 0 else None
'height': tx_height,
'confirmations': (best_height+1) - tx_height if tx_height > 0 else tx_height
}
if txo.is_change is not None:
output['is_change'] = txo.is_change
if txo.is_my_account is not None:
output['is_mine'] = txo.is_my_account
if txo.script.is_claim_name:
output['type'] = 'claim'
output['claim_op'] = 'create'
elif txo.script.is_update_claim:
output['type'] = 'claim'
output['claim_op'] = 'update'
elif txo.script.is_support_claim:
output['type'] = 'support'
else:
output['type'] = 'payment'
if txo.script.is_claim_involved:
output.update({
'name': txo.claim_name,
'claim_id': txo.claim_id,
'permanent_url': txo.permanent_url,
})
if txo.script.is_claim_name or txo.script.is_update_claim:
output['value'] = txo.claim
output['value_type'] = txo.claim.claim_type
if self.include_protobuf:
output['protobuf'] = hexlify(txo.claim.to_bytes())
if txo.channel is not None:
output['signing_channel'] = {
'name': txo.channel.claim_name,
'claim_id': txo.channel.claim_id,
'value': txo.channel.claim
}
if check_signature and txo.claim.is_signed:
output['is_channel_signature_valid'] = False
claim = txo.claim
output['value'] = claim.claim_dict
if claim.has_signature:
output['valid_signature'] = None
if txo.channel is not None:
output['channel_name'] = txo.channel.claim_name
try:
output['is_channel_signature_valid'] = txo.is_signed_by(txo.channel, self.ledger)
output['valid_signature'] = claim.validate_signature(
txo.get_address(self.ledger), txo.channel.claim, name=txo.claim_name
)
except BadSignatureError:
pass
output['valid_signature'] = False
except ValueError:
log.exception(
'txo.id: %s, txo.channel.id:%s, output: %s',
txo.id, txo.channel.id, output
)
output['valid_signature'] = False
if txo.script.is_claim_name:
output['type'] = 'claim'
elif txo.script.is_update_claim:
output['type'] = 'update'
elif txo.script.is_support_claim:
output['type'] = 'support'
else:
output['type'] = 'basic'
return output
def encode_input(self, txi):
return self.encode_output(txi.txo_ref.txo, False) if txi.txo_ref.txo is not None else {
return self.encode_output(txi.txo_ref.txo) if txi.txo_ref.txo is not None else {
'txid': txi.txo_ref.tx_ref.id,
'nout': txi.txo_ref.position
}
def encode_account(self, account):
result = account.to_dict()
result['id'] = account.id
result.pop('certificates', None)
result['is_default'] = self.ledger.accounts[0] == account
return result
def encode_file(self, managed_stream):
file = managed_stream.as_dict()
tx_height = managed_stream.stream_claim_info.height
best_height = self.ledger.headers.height
file.update({
'height': tx_height,
'confirmations': (best_height+1) - tx_height if tx_height > 0 else tx_height,
'timestamp': self.ledger.headers[tx_height]['timestamp'] if tx_height > 0 else None
})
return file
@staticmethod
def encode_claim(claim):
return getattr(claim, claim.claim_type).to_dict()
@@ -22,10 +22,6 @@ def migrate_db(conf, start, end):
from .migrate7to8 import do_migration
elif current == 8:
from .migrate8to9 import do_migration
elif current == 9:
from .migrate9to10 import do_migration
elif current == 10:
from .migrate10to11 import do_migration
else:
raise Exception("DB migration of version {} to {} is not available".format(current,
current+1))
@@ -1,54 +0,0 @@
import sqlite3
import os
import binascii
def do_migration(conf):
db_path = os.path.join(conf.data_dir, "lbrynet.sqlite")
connection = sqlite3.connect(db_path)
cursor = connection.cursor()
current_columns = []
for col_info in cursor.execute("pragma table_info('file');").fetchall():
current_columns.append(col_info[1])
if 'content_fee' in current_columns or 'saved_file' in current_columns:
connection.close()
print("already migrated")
return
cursor.execute(
"pragma foreign_keys=off;"
)
cursor.execute("""
create table if not exists new_file (
stream_hash text primary key not null references stream,
file_name text,
download_directory text,
blob_data_rate real not null,
status text not null,
saved_file integer not null,
content_fee text
);
""")
for (stream_hash, file_name, download_dir, data_rate, status) in cursor.execute("select * from file").fetchall():
saved_file = 0
if download_dir != '{stream}' and file_name != '{stream}':
try:
if os.path.isfile(os.path.join(binascii.unhexlify(download_dir).decode(),
binascii.unhexlify(file_name).decode())):
saved_file = 1
else:
download_dir, file_name = None, None
except Exception:
download_dir, file_name = None, None
else:
download_dir, file_name = None, None
cursor.execute(
"insert into new_file values (?, ?, ?, ?, ?, ?, NULL)",
(stream_hash, file_name, download_dir, data_rate, status, saved_file)
)
cursor.execute("drop table file")
cursor.execute("alter table new_file rename to file")
connection.commit()
connection.close()
@@ -3,7 +3,7 @@ import os
import json
import logging
from binascii import hexlify
from lbrynet.schema.claim import Claim
from lbrynet.schema.decode import smart_decode
log = logging.getLogger(__name__)
@@ -260,7 +260,7 @@ def do_migration(conf):
(
"%s:%i" % (claim_arg_tup[0], claim_arg_tup[1]), claim_arg_tup[2], claim_arg_tup[3],
claim_arg_tup[7], claim_arg_tup[6], claim_arg_tup[8],
Claim.from_bytes(claim_arg_tup[8]).signing_channel_id, claim_arg_tup[5], claim_arg_tup[4]
smart_decode(claim_arg_tup[8]).certificate_id, claim_arg_tup[5], claim_arg_tup[4]
)
for sd_hash, claim_arg_tup in claim_queries.items() if claim_arg_tup
] # sd_hash, (txid, nout, claim_id, name, sequence, address, height, amount, serialized)
@@ -1,20 +0,0 @@
import sqlite3
import os
def do_migration(conf):
db_path = os.path.join(conf.data_dir, "lbrynet.sqlite")
connection = sqlite3.connect(db_path)
cursor = connection.cursor()
query = "select stream_hash, sd_hash from main.stream"
for stream_hash, sd_hash in cursor.execute(query).fetchall():
head_blob_hash = cursor.execute(
"select blob_hash from stream_blob where position = 0 and stream_hash = ?",
(stream_hash,)
).fetchone()
if not head_blob_hash:
continue
cursor.execute("update blob set should_announce=1 where blob_hash in (?, ?)", (sd_hash, head_blob_hash[0],))
connection.commit()
connection.close()
+160
View File
@@ -0,0 +1,160 @@
import os
types_map = {
# http://www.iana.org/assignments/media-types
'.a': 'application/octet-stream',
'.ai': 'application/postscript',
'.aif': 'audio/x-aiff',
'.aifc': 'audio/x-aiff',
'.aiff': 'audio/x-aiff',
'.au': 'audio/basic',
'.avi': 'video/x-msvideo',
'.bat': 'text/plain',
'.bcpio': 'application/x-bcpio',
'.bin': 'application/octet-stream',
'.bmp': 'image/bmp',
'.c': 'text/plain',
'.cdf': 'application/x-netcdf',
'.cpio': 'application/x-cpio',
'.csh': 'application/x-csh',
'.css': 'text/css',
'.csv': 'text/csv',
'.dll': 'application/octet-stream',
'.doc': 'application/msword',
'.dot': 'application/msword',
'.dvi': 'application/x-dvi',
'.eml': 'message/rfc822',
'.eps': 'application/postscript',
'.epub': 'application/epub+zip',
'.etx': 'text/x-setext',
'.exe': 'application/octet-stream',
'.gif': 'image/gif',
'.gtar': 'application/x-gtar',
'.h': 'text/plain',
'.hdf': 'application/x-hdf',
'.htm': 'text/html',
'.html': 'text/html',
'.ico': 'image/vnd.microsoft.icon',
'.ief': 'image/ief',
'.iges': 'model/iges',
'.jpe': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.jpg': 'image/jpeg',
'.js': 'application/javascript',
'.json': 'application/json',
'.ksh': 'text/plain',
'.latex': 'application/x-latex',
'.m1v': 'video/mpeg',
'.m3u': 'application/vnd.apple.mpegurl',
'.m3u8': 'application/vnd.apple.mpegurl',
'.man': 'application/x-troff-man',
'.markdown': 'text/markdown',
'.md': 'text/markdown',
'.me': 'application/x-troff-me',
'.mht': 'message/rfc822',
'.mhtml': 'message/rfc822',
'.mif': 'application/x-mif',
'.mov': 'video/quicktime',
'.movie': 'video/x-sgi-movie',
'.mp2': 'audio/mpeg',
'.mp3': 'audio/mpeg',
'.mp4': 'video/mp4',
'.mpa': 'video/mpeg',
'.mpe': 'video/mpeg',
'.mpeg': 'video/mpeg',
'.mpg': 'video/mpeg',
'.ms': 'application/x-troff-ms',
'.nc': 'application/x-netcdf',
'.nws': 'message/rfc822',
'.o': 'application/octet-stream',
'.obj': 'application/octet-stream',
'.oda': 'application/oda',
'.p12': 'application/x-pkcs12',
'.p7c': 'application/pkcs7-mime',
'.pbm': 'image/x-portable-bitmap',
'.pdf': 'application/pdf',
'.pfx': 'application/x-pkcs12',
'.pgm': 'image/x-portable-graymap',
'.pl': 'text/plain',
'.png': 'image/png',
'.pnm': 'image/x-portable-anymap',
'.pot': 'application/vnd.ms-powerpoint',
'.ppa': 'application/vnd.ms-powerpoint',
'.ppm': 'image/x-portable-pixmap',
'.pps': 'application/vnd.ms-powerpoint',
'.ppt': 'application/vnd.ms-powerpoint',
'.ps': 'application/postscript',
'.pwz': 'application/vnd.ms-powerpoint',
'.py': 'text/x-python',
'.pyc': 'application/x-python-code',
'.pyo': 'application/x-python-code',
'.qt': 'video/quicktime',
'.ra': 'audio/x-pn-realaudio',
'.ram': 'application/x-pn-realaudio',
'.ras': 'image/x-cmu-raster',
'.rdf': 'application/xml',
'.rgb': 'image/x-rgb',
'.roff': 'application/x-troff',
'.rtx': 'text/richtext',
'.sgm': 'text/x-sgml',
'.sgml': 'text/x-sgml',
'.sh': 'application/x-sh',
'.shar': 'application/x-shar',
'.snd': 'audio/basic',
'.so': 'application/octet-stream',
'.src': 'application/x-wais-source',
'.stl': 'model/stl',
'.sv4cpio': 'application/x-sv4cpio',
'.sv4crc': 'application/x-sv4crc',
'.svg': 'image/svg+xml',
'.swf': 'application/x-shockwave-flash',
'.t': 'application/x-troff',
'.tar': 'application/x-tar',
'.tcl': 'application/x-tcl',
'.tex': 'application/x-tex',
'.texi': 'application/x-texinfo',
'.texinfo': 'application/x-texinfo',
'.tif': 'image/tiff',
'.tiff': 'image/tiff',
'.tr': 'application/x-troff',
'.tsv': 'text/tab-separated-values',
'.txt': 'text/plain',
'.ustar': 'application/x-ustar',
'.vcf': 'text/x-vcard',
'.wav': 'audio/x-wav',
'.webm': 'video/webm',
'.wiz': 'application/msword',
'.wsdl': 'application/xml',
'.xbm': 'image/x-xbitmap',
'.xlb': 'application/vnd.ms-excel',
'.xls': 'application/vnd.ms-excel',
'.xml': 'text/xml',
'.xpdl': 'application/xml',
'.xpm': 'image/x-xpixmap',
'.xsl': 'application/xml',
'.xwd': 'image/x-xwindowdump',
'.zip': 'application/zip',
# These are non-standard types, commonly found in the wild.
'.mid': 'audio/midi',
'.midi': 'audio/midi',
'.pct': 'image/pict',
'.pic': 'image/pict',
'.pict': 'image/pict',
'.rtf': 'application/rtf',
'.xul': 'text/xul',
'.m4v': 'video/m4v',
# microsoft is special and has its own 'standard'
# https://docs.microsoft.com/en-us/windows/desktop/wmp/file-name-extensions
'.wmv': 'video/x-ms-wmv'
}
def guess_media_type(path):
_, ext = os.path.splitext(path)
return types_map.get(
ext.strip().lower(),
'application/octet-stream' if not (ext and ext[1:].strip()) else f'application/x-ext-{ext[1:].strip().lower()}'
)
+96 -171
View File
@@ -7,9 +7,9 @@ import binascii
import time
from torba.client.basedatabase import SQLiteMixin
from lbrynet.conf import Config
from lbrynet.wallet.dewies import dewies_to_lbc, lbc_to_dewies
from lbrynet.wallet.transaction import Transaction
from lbrynet.schema.claim import Claim
from lbrynet.extras.wallet.dewies import dewies_to_lbc, lbc_to_dewies
from lbrynet.schema.claim import ClaimDict
from lbrynet.schema.decode import smart_decode
from lbrynet.dht.constants import data_expiration
from lbrynet.blob.blob_info import BlobInfo
@@ -39,9 +39,7 @@ class StoredStreamClaim:
self.claim_name = name
self.amount = amount
self.height = height
self.claim: typing.Optional[Claim] = None if not serialized else Claim.from_bytes(
binascii.unhexlify(serialized)
)
self.claim: typing.Optional[ClaimDict] = None if not serialized else smart_decode(serialized)
self.claim_address = address
self.claim_sequence = claim_sequence
self.channel_claim_id = channel_claim_id
@@ -55,6 +53,10 @@ class StoredStreamClaim:
def nout(self) -> typing.Optional[int]:
return None if not self.outpoint else int(self.outpoint.split(":")[1])
@property
def metadata(self) -> typing.Optional[typing.Dict]:
return None if not self.claim else self.claim.claim_dict['stream']['metadata']
def as_dict(self) -> typing.Dict:
return {
"name": self.claim_name,
@@ -115,8 +117,8 @@ def _batched_select(transaction, query, parameters, batch_size=900):
def get_all_lbry_files(transaction: sqlite3.Connection) -> typing.List[typing.Dict]:
files = []
signed_claims = {}
for (rowid, stream_hash, file_name, download_dir, data_rate, status, saved_file, raw_content_fee, _,
sd_hash, stream_key, stream_name, suggested_file_name, *claim_args) in _batched_select(
for (rowid, stream_hash, file_name, download_dir, data_rate, status, _, sd_hash, stream_key,
stream_name, suggested_file_name, *claim_args) in _batched_select(
transaction, "select file.rowid, file.*, stream.*, c.* "
"from file inner join stream on file.stream_hash=stream.stream_hash "
"inner join content_claim cc on file.stream_hash=cc.stream_hash "
@@ -142,11 +144,7 @@ def get_all_lbry_files(transaction: sqlite3.Connection) -> typing.List[typing.Di
"key": stream_key,
"stream_name": stream_name, # hex
"suggested_file_name": suggested_file_name, # hex
"claim": claim,
"saved_file": bool(saved_file),
"content_fee": None if not raw_content_fee else Transaction(
binascii.unhexlify(raw_content_fee)
)
"claim": claim
}
)
for claim_name, claim_id in _batched_select(
@@ -158,12 +156,21 @@ def get_all_lbry_files(transaction: sqlite3.Connection) -> typing.List[typing.Di
def store_stream(transaction: sqlite3.Connection, sd_blob: 'BlobFile', descriptor: 'StreamDescriptor'):
# add all blobs, except the last one, which is empty
transaction.executemany(
"insert or ignore into blob values (?, ?, ?, ?, ?, ?, ?)",
[(blob.blob_hash, blob.length, 0, 0, "pending", 0, 0)
for blob in (descriptor.blobs[:-1] if len(descriptor.blobs) > 1 else descriptor.blobs) + [sd_blob]]
# add the head blob and set it to be announced
transaction.execute(
"insert or ignore into blob values (?, ?, ?, ?, ?, ?, ?), (?, ?, ?, ?, ?, ?, ?)",
(
sd_blob.blob_hash, sd_blob.length, 0, 1, "pending", 0, 0,
descriptor.blobs[0].blob_hash, descriptor.blobs[0].length, 0, 1, "pending", 0, 0
)
)
# add the rest of the blobs with announcement off
if len(descriptor.blobs) > 2:
transaction.executemany(
"insert or ignore into blob values (?, ?, ?, ?, ?, ?, ?)",
[(blob.blob_hash, blob.length, 0, 0, "pending", 0, 0)
for blob in descriptor.blobs[1:-1]]
)
# associate the blobs to the stream
transaction.execute("insert or ignore into stream values (?, ?, ?, ?, ?)",
(descriptor.stream_hash, sd_blob.blob_hash, descriptor.key,
@@ -175,11 +182,6 @@ def store_stream(transaction: sqlite3.Connection, sd_blob: 'BlobFile', descripto
[(descriptor.stream_hash, blob.blob_hash, blob.blob_num, blob.iv)
for blob in descriptor.blobs]
)
# ensure should_announce is set regardless if insert was ignored
transaction.execute(
"update blob set should_announce=1 where blob_hash in (?, ?)",
(sd_blob.blob_hash, descriptor.blobs[0].blob_hash,)
)
def delete_stream(transaction: sqlite3.Connection, descriptor: 'StreamDescriptor'):
@@ -192,21 +194,13 @@ def delete_stream(transaction: sqlite3.Connection, descriptor: 'StreamDescriptor
transaction.executemany("delete from blob where blob_hash=?", blob_hashes)
def store_file(transaction: sqlite3.Connection, stream_hash: str, file_name: typing.Optional[str],
download_directory: typing.Optional[str], data_payment_rate: float, status: str,
content_fee: typing.Optional[Transaction]) -> int:
if not file_name and not download_directory:
encoded_file_name, encoded_download_dir = None, None
else:
encoded_file_name = binascii.hexlify(file_name.encode()).decode()
encoded_download_dir = binascii.hexlify(download_directory.encode()).decode()
def store_file(transaction: sqlite3.Connection, stream_hash: str, file_name: str, download_directory: str,
data_payment_rate: float, status: str) -> int:
transaction.execute(
"insert or replace into file values (?, ?, ?, ?, ?, ?, ?)",
(stream_hash, encoded_file_name, encoded_download_dir, data_payment_rate, status,
1 if (file_name and download_directory and os.path.isfile(os.path.join(download_directory, file_name))) else 0,
None if not content_fee else binascii.hexlify(content_fee.raw).decode())
"insert or replace into file values (?, ?, ?, ?, ?)",
(stream_hash, binascii.hexlify(file_name.encode()).decode(),
binascii.hexlify(download_directory.encode()).decode(), data_payment_rate, status)
)
return transaction.execute("select rowid from file where stream_hash=?", (stream_hash, )).fetchone()[0]
@@ -255,12 +249,10 @@ class SQLiteStorage(SQLiteMixin):
create table if not exists file (
stream_hash text primary key not null references stream,
file_name text,
download_directory text,
file_name text not null,
download_directory text not null,
blob_data_rate real not null,
status text not null,
saved_file integer not null,
content_fee text
status text not null
);
create table if not exists content_claim (
@@ -303,28 +295,27 @@ class SQLiteStorage(SQLiteMixin):
# # # # # # # # # blob functions # # # # # # # # #
async def add_blobs(self, *blob_hashes_and_lengths: typing.Tuple[str, int], finished=False):
def _add_blobs(transaction: sqlite3.Connection):
transaction.executemany(
def add_completed_blob(self, blob_hash: str, length: int):
def _add_blob(transaction: sqlite3.Connection):
transaction.execute(
"insert or ignore into blob values (?, ?, ?, ?, ?, ?, ?)",
[
(blob_hash, length, 0, 0, "pending" if not finished else "finished", 0, 0)
for blob_hash, length in blob_hashes_and_lengths
]
(blob_hash, length, 0, 0, "pending", 0, 0)
)
if finished:
transaction.executemany(
"update blob set status='finished' where blob.blob_hash=?", [
(blob_hash, ) for blob_hash, _ in blob_hashes_and_lengths
]
)
return await self.db.run(_add_blobs)
transaction.execute(
"update blob set status='finished' where blob.blob_hash=?", (blob_hash, )
)
return self.db.run(_add_blob)
def get_blob_status(self, blob_hash: str):
return self.run_and_return_one_or_none(
"select status from blob where blob_hash=?", blob_hash
)
def add_known_blob(self, blob_hash: str, length: int):
return self.db.execute(
"insert or ignore into blob values (?, ?, ?, ?, ?, ?, ?)", (blob_hash, length, 0, 0, "pending", 0, 0)
)
def should_announce(self, blob_hash: str):
return self.run_and_return_one_or_none(
"select should_announce from blob where blob_hash=?", blob_hash
@@ -427,26 +418,6 @@ class SQLiteStorage(SQLiteMixin):
}
return self.db.run(_sync_blobs)
def sync_files_to_blobs(self):
def _sync_blobs(transaction: sqlite3.Connection):
transaction.executemany(
"update file set status='stopped' where stream_hash=?",
transaction.execute(
"select distinct sb.stream_hash from stream_blob sb "
"inner join blob b on b.blob_hash=sb.blob_hash and b.status=='pending'"
).fetchall()
)
return self.db.run(_sync_blobs)
def set_files_as_streaming(self, stream_hashes: typing.List[str]):
def _set_streaming(transaction: sqlite3.Connection):
transaction.executemany(
"update file set file_name=null, download_directory=null where stream_hash=?",
[(stream_hash, ) for stream_hash in stream_hashes]
)
return self.db.run(_set_streaming)
# # # # # # # # # stream functions # # # # # # # # #
async def stream_exists(self, sd_hash: str) -> bool:
@@ -509,53 +480,20 @@ class SQLiteStorage(SQLiteMixin):
"select stream_hash from stream where sd_hash = ?", sd_blob_hash
)
def get_stream_info_for_sd_hash(self, sd_blob_hash):
return self.run_and_return_one_or_none(
"select stream_hash, stream_name, suggested_filename, stream_key from stream where sd_hash = ?",
sd_blob_hash
)
def delete_stream(self, descriptor: 'StreamDescriptor'):
return self.db.run_with_foreign_keys_disabled(delete_stream, descriptor)
# # # # # # # # # file stuff # # # # # # # # #
def save_downloaded_file(self, stream_hash: str, file_name: typing.Optional[str],
download_directory: typing.Optional[str], data_payment_rate: float,
content_fee: typing.Optional[Transaction] = None) -> typing.Awaitable[int]:
def save_downloaded_file(self, stream_hash, file_name, download_directory,
data_payment_rate) -> typing.Awaitable[int]:
return self.save_published_file(
stream_hash, file_name, download_directory, data_payment_rate, status="running",
content_fee=content_fee
stream_hash, file_name, download_directory, data_payment_rate, status="running"
)
def save_published_file(self, stream_hash: str, file_name: typing.Optional[str],
download_directory: typing.Optional[str], data_payment_rate: float,
status: str = "finished",
content_fee: typing.Optional[Transaction] = None) -> typing.Awaitable[int]:
return self.db.run(store_file, stream_hash, file_name, download_directory, data_payment_rate, status,
content_fee)
async def update_manually_removed_files_since_last_run(self):
"""
Update files that have been removed from the downloads directory since the last run
"""
def update_manually_removed_files(transaction: sqlite3.Connection):
removed = []
for (stream_hash, download_directory, file_name) in transaction.execute(
"select stream_hash, download_directory, file_name from file where saved_file=1"
).fetchall():
if download_directory and file_name and os.path.isfile(
os.path.join(binascii.unhexlify(download_directory).decode(),
binascii.unhexlify(file_name).decode())):
continue
else:
removed.append((stream_hash,))
if removed:
transaction.executemany(
"update file set file_name=null, download_directory=null, saved_file=0 where stream_hash=?",
removed
)
return await self.db.run(update_manually_removed_files)
def save_published_file(self, stream_hash: str, file_name: str, download_directory: str, data_payment_rate: float,
status="finished") -> typing.Awaitable[int]:
return self.db.run(store_file, stream_hash, file_name, download_directory, data_payment_rate, status)
def get_all_lbry_files(self) -> typing.Awaitable[typing.List[typing.Dict]]:
return self.db.run(get_all_lbry_files)
@@ -564,45 +502,24 @@ class SQLiteStorage(SQLiteMixin):
log.debug("update file status %s -> %s", stream_hash, new_status)
return self.db.execute("update file set status=? where stream_hash=?", (new_status, stream_hash))
async def change_file_download_dir_and_file_name(self, stream_hash: str, download_dir: typing.Optional[str],
file_name: typing.Optional[str]):
if not file_name or not download_dir:
encoded_file_name, encoded_download_dir = None, None
else:
encoded_file_name = binascii.hexlify(file_name.encode()).decode()
encoded_download_dir = binascii.hexlify(download_dir.encode()).decode()
return await self.db.execute("update file set download_directory=?, file_name=? where stream_hash=?", (
encoded_download_dir, encoded_file_name, stream_hash,
def change_file_download_dir_and_file_name(self, stream_hash: str, download_dir: str, file_name: str):
return self.db.execute("update file set download_directory=?, file_name=? where stream_hash=?", (
binascii.hexlify(download_dir.encode()).decode(), binascii.hexlify(file_name.encode()).decode(),
stream_hash
))
async def save_content_fee(self, stream_hash: str, content_fee: Transaction):
return await self.db.execute("update file set content_fee=? where stream_hash=?", (
binascii.hexlify(content_fee.raw), stream_hash,
))
async def set_saved_file(self, stream_hash: str):
return await self.db.execute("update file set saved_file=1 where stream_hash=?", (
stream_hash,
))
async def clear_saved_file(self, stream_hash: str):
return await self.db.execute("update file set saved_file=0 where stream_hash=?", (
stream_hash,
))
async def recover_streams(self, descriptors_and_sds: typing.List[typing.Tuple['StreamDescriptor', 'BlobFile',
typing.Optional[Transaction]]],
async def recover_streams(self, descriptors_and_sds: typing.List[typing.Tuple['StreamDescriptor', 'BlobFile']],
download_directory: str):
def _recover(transaction: sqlite3.Connection):
stream_hashes = [x[0].stream_hash for x in descriptors_and_sds]
for descriptor, sd_blob, content_fee in descriptors_and_sds:
stream_hashes = [d.stream_hash for d, s in descriptors_and_sds]
for descriptor, sd_blob in descriptors_and_sds:
content_claim = transaction.execute(
"select * from content_claim where stream_hash=?", (descriptor.stream_hash, )
).fetchone()
delete_stream(transaction, descriptor) # this will also delete the content claim
store_stream(transaction, sd_blob, descriptor)
store_file(transaction, descriptor.stream_hash, os.path.basename(descriptor.suggested_file_name),
download_directory, 0.0, 'stopped', content_fee=content_fee)
download_directory, 0.0, 'stopped')
if content_claim:
transaction.execute("insert or ignore into content_claim values (?, ?)", content_claim)
transaction.executemany(
@@ -621,18 +538,16 @@ class SQLiteStorage(SQLiteMixin):
# # # # # # # # # support functions # # # # # # # # #
def save_supports(self, claim_id_to_supports: dict):
def save_supports(self, claim_id, supports):
# TODO: add 'address' to support items returned for a claim from lbrycrdd and lbryum-server
def _save_support(transaction):
bind = "({})".format(','.join(['?'] * len(claim_id_to_supports)))
transaction.execute(f"delete from support where claim_id in {bind}", list(claim_id_to_supports.keys()))
for claim_id, supports in claim_id_to_supports.items():
for support in supports:
transaction.execute(
"insert into support values (?, ?, ?, ?)",
("%s:%i" % (support['txid'], support['nout']), claim_id, lbc_to_dewies(support['amount']),
support.get('address', ""))
)
transaction.execute("delete from support where claim_id=?", (claim_id,))
for support in supports:
transaction.execute(
"insert into support values (?, ?, ?, ?)",
("%s:%i" % (support['txid'], support['nout']), claim_id, lbc_to_dewies(support['amount']),
support.get('address', ""))
)
return self.db.run(_save_support)
def get_supports(self, *claim_ids):
@@ -660,7 +575,7 @@ class SQLiteStorage(SQLiteMixin):
# # # # # # # # # claim functions # # # # # # # # #
async def save_claims(self, claim_infos):
claim_id_to_supports = {}
support_callbacks = []
update_file_callbacks = []
def _save_claims(transaction):
@@ -673,12 +588,19 @@ class SQLiteStorage(SQLiteMixin):
height = claim_info['height']
address = claim_info['address']
sequence = claim_info['claim_sequence']
certificate_id = claim_info['value'].signing_channel_id
try:
source_hash = claim_info['value'].stream.source.sd_hash
except (AttributeError, ValueError):
certificate_id = claim_info['value'].get('publisherSignature', {}).get('certificateId')
except AttributeError:
certificate_id = None
try:
if claim_info['value'].get('stream', {}).get('source', {}).get('sourceType') == "lbry_sd_hash":
source_hash = claim_info['value'].get('stream', {}).get('source', {}).get('source')
else:
source_hash = None
except AttributeError:
source_hash = None
serialized = binascii.hexlify(claim_info['value'].to_bytes())
serialized = claim_info.get('hex') or binascii.hexlify(
smart_decode(claim_info['value']).serialized).decode()
transaction.execute(
"insert or replace into claim values (?, ?, ?, ?, ?, ?, ?, ?, ?)",
(outpoint, claim_id, name, amount, height, serialized, certificate_id, address, sequence)
@@ -686,7 +608,7 @@ class SQLiteStorage(SQLiteMixin):
# if this response doesn't have support info don't overwrite the existing
# support info
if 'supports' in claim_info:
claim_id_to_supports[claim_id] = claim_info['supports']
support_callbacks.append((claim_id, claim_info['supports']))
if not source_hash:
continue
stream_hash = transaction.execute(
@@ -698,12 +620,12 @@ class SQLiteStorage(SQLiteMixin):
stream_hash = stream_hash[0]
known_outpoint = transaction.execute(
"select claim_outpoint from content_claim where stream_hash=?", (stream_hash,)
).fetchone()
)
known_claim_id = transaction.execute(
"select claim_id from claim "
"inner join content_claim c3 ON claim.claim_outpoint=c3.claim_outpoint "
"where c3.stream_hash=?", (stream_hash,)
).fetchone()
)
if not known_claim_id:
content_claims_to_update.append((stream_hash, outpoint))
elif known_outpoint != outpoint:
@@ -716,20 +638,23 @@ class SQLiteStorage(SQLiteMixin):
await self.db.run(_save_claims)
if update_file_callbacks:
await asyncio.wait(update_file_callbacks)
if claim_id_to_supports:
await self.save_supports(claim_id_to_supports)
if support_callbacks:
await asyncio.wait([
self.save_supports(*args) for args in support_callbacks
])
def save_claims_for_resolve(self, claim_infos):
to_save = {}
to_save = []
for info in claim_infos:
if 'value' in info:
if info['value']:
to_save[info['claim_id']] = info
to_save.append(info)
else:
for key in ('certificate', 'claim'):
if info.get(key, {}).get('value'):
to_save[info[key]['claim_id']] = info[key]
return self.save_claims(to_save.values())
if 'certificate' in info and info['certificate']['value']:
to_save.append(info['certificate'])
if 'claim' in info and info['claim']['value']:
to_save.append(info['claim'])
return self.save_claims(to_save)
@staticmethod
def _save_content_claim(transaction, claim_outpoint, stream_hash):
@@ -739,7 +664,7 @@ class SQLiteStorage(SQLiteMixin):
).fetchone()
if not claim_info:
raise Exception("claim not found")
new_claim_id, claim = claim_info[0], Claim.from_bytes(binascii.unhexlify(claim_info[1]))
new_claim_id, claim = claim_info[0], ClaimDict.deserialize(binascii.unhexlify(claim_info[1]))
# certificate claims should not be in the content_claim table
if not claim.is_stream:
@@ -752,7 +677,7 @@ class SQLiteStorage(SQLiteMixin):
if not known_sd_hash:
raise Exception("stream not found")
# check the claim contains the same sd hash
if known_sd_hash[0] != claim.stream.source.sd_hash:
if known_sd_hash[0].encode() != claim.source_hash:
raise Exception("stream mismatch")
# if there is a current claim associated to the file, check that the new claim is an update to it
+2
View File
@@ -2,6 +2,7 @@ import platform
import os
import logging.handlers
from lbrynet.schema import __version__ as schema_version
from lbrynet import build_type, __version__ as lbrynet_version
log = logging.getLogger(__name__)
@@ -15,6 +16,7 @@ def get_platform() -> dict:
"os_release": platform.release(),
"os_system": platform.system(),
"lbrynet_version": lbrynet_version,
"lbryschema_version": schema_version,
"build": build_type.BUILD, # CI server sets this during build step
}
if p["os_system"] == "Linux":
+11
View File
@@ -0,0 +1,11 @@
__node_daemon__ = 'lbrycrdd'
__node_cli__ = 'lbrycrd-cli'
__node_bin__ = ''
__node_url__ = (
'https://github.com/lbryio/lbrycrd/releases/download/v0.12.4.0/lbrycrd-linux.zip'
)
__spvserver__ = 'lbrynet.extras.wallet.server.coin.LBCRegTest'
from lbrynet.extras.wallet.ledger import MainNetLedger, RegTestLedger
from lbrynet.extras.wallet.manager import LbryWalletManager
from lbrynet.extras.wallet.network import Network
@@ -2,50 +2,49 @@ import json
import logging
import binascii
from hashlib import sha256
from string import hexdigits
from torba.client.baseaccount import BaseAccount, HierarchicalDeterministic
from lbrynet.schema.validator import validate_claim_id
from torba.client.baseaccount import BaseAccount
from torba.client.basetransaction import TXORef
from lbrynet.schema.claim import ClaimDict
from lbrynet.schema.signer import SECP256k1, get_signer
log = logging.getLogger(__name__)
def validate_claim_id(claim_id):
if not len(claim_id) == 40:
raise Exception("Incorrect claimid length: %i" % len(claim_id))
if isinstance(claim_id, bytes):
claim_id = claim_id.decode('utf-8')
if set(claim_id).difference(hexdigits):
raise Exception("Claim id is not hex encoded")
def generate_certificate():
secp256k1_private_key = get_signer(SECP256k1).generate().private_key.to_pem()
return ClaimDict.generate_certificate(secp256k1_private_key, curve=SECP256k1), secp256k1_private_key
class Account(BaseAccount):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.channel_keys = {}
self.certificates = {}
@property
def hash(self) -> bytes:
h = sha256(json.dumps(self.to_dict(False)).encode())
for cert in sorted(self.channel_keys.keys()):
for cert in sorted(self.certificates.keys()):
h.update(cert.encode())
return h.digest()
def apply(self, d: dict):
super().apply(d)
self.channel_keys.update(d.get('certificates', {}))
self.certificates.update(d.get('certificates', {}))
def add_channel_private_key(self, ref: TXORef, private_key):
assert ref.id not in self.channel_keys, 'Trying to add a duplicate channel private key.'
self.channel_keys[ref.id] = private_key
def add_certificate_private_key(self, ref: TXORef, private_key):
assert ref.id not in self.certificates, 'Trying to add a duplicate certificate.'
self.certificates[ref.id] = private_key
def get_channel_private_key(self, ref: TXORef):
return self.channel_keys.get(ref.id)
def get_certificate_private_key(self, ref: TXORef):
return self.certificates.get(ref.id)
async def maybe_migrate_certificates(self):
if not self.channel_keys:
if not self.certificates:
return
addresses = {}
@@ -59,7 +58,7 @@ class Account(BaseAccount):
}
double_hex_encoded_to_pop = []
for maybe_claim_id in list(self.channel_keys):
for maybe_claim_id in list(self.certificates):
if ':' not in maybe_claim_id:
try:
validate_claim_id(maybe_claim_id)
@@ -71,7 +70,7 @@ class Account(BaseAccount):
maybe_claim_id_bytes = maybe_claim_id_bytes.encode()
decoded_double_hex = binascii.unhexlify(maybe_claim_id_bytes).decode()
validate_claim_id(decoded_double_hex)
if decoded_double_hex in self.channel_keys:
if decoded_double_hex in self.certificates:
log.warning("don't know how to migrate certificate %s", decoded_double_hex)
else:
log.info("claim id was double hex encoded, fixing it")
@@ -80,9 +79,9 @@ class Account(BaseAccount):
continue
for double_encoded_claim_id, correct_claim_id in double_hex_encoded_to_pop:
self.channel_keys[correct_claim_id] = self.channel_keys.pop(double_encoded_claim_id)
self.certificates[correct_claim_id] = self.certificates.pop(double_encoded_claim_id)
for maybe_claim_id in list(self.channel_keys):
for maybe_claim_id in list(self.certificates):
results['total'] += 1
if ':' not in maybe_claim_id:
try:
@@ -117,8 +116,8 @@ class Account(BaseAccount):
.format(maybe_claim_id)
)
tx_nout = '{txid}:{nout}'.format(**claim)
self.channel_keys[tx_nout] = self.channel_keys[maybe_claim_id]
del self.channel_keys[maybe_claim_id]
self.certificates[tx_nout] = self.certificates[maybe_claim_id]
del self.certificates[maybe_claim_id]
log.info(
"Migrated certificate with claim_id '%s' ('%s') to a new look up key %s.",
maybe_claim_id, txo.script.values['claim_name'], tx_nout
@@ -167,11 +166,10 @@ class Account(BaseAccount):
))
async def save_max_gap(self):
if issubclass(self.address_generator, HierarchicalDeterministic):
gap = await self.get_max_gap()
self.receiving.gap = max(20, gap['max_receiving_gap'] + 1)
self.change.gap = max(6, gap['max_change_gap'] + 1)
self.wallet.save()
gap = await self.get_max_gap()
self.receiving.gap = max(20, gap['max_receiving_gap'] + 1)
self.change.gap = max(6, gap['max_change_gap'] + 1)
self.wallet.save()
def get_balance(self, confirmations=0, include_claims=False, **constraints):
if not include_claims:
@@ -179,7 +177,7 @@ class Account(BaseAccount):
return super().get_balance(confirmations, **constraints)
@classmethod
def get_private_key_from_seed(cls, ledger, seed: str, password: str):
def get_private_key_from_seed(cls, ledger: 'baseledger.BaseLedger', seed: str, password: str):
return super().get_private_key_from_seed(
ledger, seed, password or 'lbryum'
)
@@ -187,30 +185,36 @@ class Account(BaseAccount):
@classmethod
def from_dict(cls, ledger, wallet, d: dict) -> 'Account':
account = super().from_dict(ledger, wallet, d)
account.channel_keys = d.get('certificates', {})
account.certificates = d.get('certificates', {})
return account
def to_dict(self, include_channel_keys=True):
def to_dict(self, include_certificates=True):
d = super().to_dict()
if include_channel_keys:
d['certificates'] = self.channel_keys
if include_certificates:
d['certificates'] = self.certificates
return d
async def get_details(self, **kwargs):
details = await super().get_details(**kwargs)
details['certificates'] = len(self.channel_keys)
details['certificates'] = len(self.certificates)
return details
def get_claim(self, claim_id=None, txid=None, nout=None):
if claim_id is not None:
return self.ledger.db.get_claims(account=self, claim_id=claim_id)
elif txid is not None and nout is not None:
return self.ledger.db.get_claims(**{'account': self, 'txo.txid': txid, 'txo.position': nout})
@staticmethod
def constraint_spending_utxos(constraints):
def constraint_utxos_sans_claims(constraints):
constraints.update({'is_claim': 0, 'is_update': 0, 'is_support': 0})
def get_utxos(self, **constraints):
self.constraint_spending_utxos(constraints)
self.constraint_utxos_sans_claims(constraints)
return super().get_utxos(**constraints)
def get_utxo_count(self, **constraints):
self.constraint_spending_utxos(constraints)
self.constraint_utxos_sans_claims(constraints)
return super().get_utxo_count(**constraints)
def get_claims(self, **constraints):
@@ -219,23 +223,32 @@ class Account(BaseAccount):
def get_claim_count(self, **constraints):
return self.ledger.db.get_claim_count(account=self, **constraints)
def get_streams(self, **constraints):
return self.ledger.db.get_streams(account=self, **constraints)
def get_stream_count(self, **constraints):
return self.ledger.db.get_stream_count(account=self, **constraints)
def get_channels(self, **constraints):
return self.ledger.db.get_channels(account=self, **constraints)
def get_channel_count(self, **constraints):
return self.ledger.db.get_channel_count(account=self, **constraints)
def get_supports(self, **constraints):
return self.ledger.db.get_supports(account=self, **constraints)
async def send_to_addresses(self, amount, addresses, broadcast=False):
tx_class = self.ledger.transaction_class
tx = await tx_class.create(
inputs=[],
outputs=[
tx_class.output_class.pay_pubkey_hash(amount, self.ledger.address_to_hash160(address))
for address in addresses
],
funding_accounts=[self],
change_account=self
)
def get_support_count(self, **constraints):
return self.ledger.db.get_support_count(account=self, **constraints)
if broadcast:
await self.ledger.broadcast(tx)
else:
await self.ledger.release_outputs(
[txi.txo_ref.txo for txi in tx.inputs]
)
return tx
async def release_all_outputs(self):
await self.ledger.db.release_all_outputs(self)
File renamed without changes.
@@ -1,9 +1,5 @@
from typing import List
from torba.client.basedatabase import BaseDatabase
from lbrynet.wallet.transaction import Output
class WalletDatabase(BaseDatabase):
@@ -52,7 +48,7 @@ class WalletDatabase(BaseDatabase):
row['claim_name'] = txo.claim_name
return row
async def get_txos(self, **constraints) -> List[Output]:
async def get_txos(self, **constraints):
my_account = constraints.get('my_account', constraints.get('account', None))
txos = await super().get_txos(**constraints)
@@ -60,10 +56,10 @@ class WalletDatabase(BaseDatabase):
channel_ids = set()
for txo in txos:
if txo.script.is_claim_name or txo.script.is_update_claim:
if txo.claim.is_signed:
channel_ids.add(txo.claim.signing_channel_id)
if txo.claim.is_channel and my_account is not None:
txo.private_key = my_account.get_channel_private_key(txo.ref)
if 'publisherSignature' in txo.claim_dict:
channel_ids.add(txo.claim_dict['publisherSignature']['certificateId'])
if txo.claim_name.startswith('@') and my_account is not None:
txo.private_key = my_account.get_certificate_private_key(txo.ref)
if channel_ids:
channels = {
@@ -75,35 +71,23 @@ class WalletDatabase(BaseDatabase):
}
for txo in txos:
if txo.script.is_claim_name or txo.script.is_update_claim:
txo.channel = channels.get(txo.claim.signing_channel_id, None)
if 'publisherSignature' in txo.claim_dict:
txo.channel = channels.get(txo.claim_dict['publisherSignature']['certificateId'])
return txos
@staticmethod
def constrain_claims(constraints):
constraints['claim_type__any'] = {'is_claim': 1, 'is_update': 1}
constraints['claim_type__any'] = {'is_claim': 1, 'is_update': 1, 'is_support': 1}
async def get_claims(self, **constraints) -> List[Output]:
def get_claims(self, **constraints):
self.constrain_claims(constraints)
return await self.get_utxos(**constraints)
return self.get_utxos(**constraints)
def get_claim_count(self, **constraints):
self.constrain_claims(constraints)
return self.get_utxo_count(**constraints)
@staticmethod
def constrain_streams(constraints):
if 'claim_name' not in constraints or 'claim_id' not in constraints:
constraints['claim_name__not_like'] = '@%'
def get_streams(self, **constraints):
self.constrain_streams(constraints)
return self.get_claims(**constraints)
def get_stream_count(self, **constraints):
self.constrain_streams(constraints)
return self.get_claim_count(**constraints)
@staticmethod
def constrain_channels(constraints):
if 'claim_name' not in constraints or 'claim_id' not in constraints:
@@ -117,17 +101,22 @@ class WalletDatabase(BaseDatabase):
self.constrain_channels(constraints)
return self.get_claim_count(**constraints)
@staticmethod
def constrain_supports(constraints):
constraints['is_support'] = 1
def get_supports(self, **constraints):
self.constrain_supports(constraints)
return self.get_utxos(**constraints)
def get_support_count(self, **constraints):
self.constrain_supports(constraints)
return self.get_utxo_count(**constraints)
async def get_certificates(self, private_key_accounts, exclude_without_key=False, **constraints):
channels = await self.get_channels(**constraints)
certificates = []
if private_key_accounts is not None:
for channel in channels:
if not channel.has_private_key:
private_key = None
for account in private_key_accounts:
private_key = account.get_certificate_private_key(channel.ref)
if private_key is not None:
break
if private_key is None and exclude_without_key:
continue
channel.private_key = private_key
certificates.append(channel)
return certificates
async def release_all_outputs(self, account):
await self.db.execute(
File renamed without changes.
File renamed without changes.
@@ -2,14 +2,17 @@ import asyncio
import logging
from binascii import unhexlify
from lbrynet.schema.validator import validate_claim_id
from torba.client.baseledger import BaseLedger
from lbrynet.wallet.dewies import dewies_to_lbc
from lbrynet.wallet.resolve import Resolver
from lbrynet.wallet.account import Account
from lbrynet.wallet.network import Network
from lbrynet.wallet.database import WalletDatabase
from lbrynet.wallet.transaction import Transaction
from lbrynet.wallet.header import Headers, UnvalidatedHeaders
from lbrynet.schema.error import URIParseError
from lbrynet.schema.uri import parse_lbry_uri
from lbrynet.extras.wallet.dewies import dewies_to_lbc
from lbrynet.extras.wallet.resolve import Resolver
from lbrynet.extras.wallet.account import Account
from lbrynet.extras.wallet.network import Network
from lbrynet.extras.wallet.database import WalletDatabase
from lbrynet.extras.wallet.transaction import Transaction
from lbrynet.extras.wallet.header import Headers, UnvalidatedHeaders
log = logging.getLogger(__name__)
@@ -28,8 +31,6 @@ class MainNetLedger(BaseLedger):
network_class = Network
transaction_class = Transaction
db: WalletDatabase
secret_prefix = bytes((0x1c,))
pubkey_address_prefix = bytes((0x55,))
script_address_prefix = bytes((0x7a,))
@@ -47,20 +48,38 @@ class MainNetLedger(BaseLedger):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fee_per_name_char = self.config.get('fee_per_name_char', self.default_fee_per_name_char)
self.resolver = Resolver(self)
def resolve(self, page, page_size, *uris):
return self.resolver.resolve(page, page_size, *uris)
@property
def resolver(self):
return Resolver(self.headers.claim_trie_root, self.headers.height, self.transaction_class,
hash160_to_address=self.hash160_to_address, network=self.network)
async def resolve(self, page, page_size, *uris):
for uri in uris:
try:
parsed_uri = parse_lbry_uri(uri)
if parsed_uri.claim_id:
validate_claim_id(parsed_uri.claim_id)
except URIParseError as err:
return {'error': err.args[0]}
except Exception as e:
return {'error': str(e)}
try:
resolver = self.resolver
resolutions = await self.network.get_values_for_uris(self.headers.hash().decode(), *uris)
return await resolver._handle_resolutions(resolutions, uris, page, page_size)
except Exception as e:
return {'error': str(e)}
async def get_claim_by_claim_id(self, claim_id):
result = (await self.network.get_claims_by_ids(claim_id)).pop(claim_id, {})
return await self.resolver.parse_and_validate_claim_result(result)
return await self.resolver.get_certificate_and_validate_result(result)
async def get_claim_by_outpoint(self, txid, nout):
claims = (await self.network.get_claims_in_tx(txid)) or []
for claim in claims:
if claim['nout'] == nout:
return await self.resolver.parse_and_validate_claim_result(claim)
return await self.resolver.get_certificate_and_validate_result(claim)
return 'claim not found'
async def start(self):
@@ -79,7 +98,7 @@ class MainNetLedger(BaseLedger):
log.info("Loaded account %s with %s LBC, %d receiving addresses (gap: %d), "
"%d change addresses (gap: %d), %d channels, %d certificates and %d claims. ",
account.id, balance, total_receiving, account.receiving.gap, total_change, account.change.gap,
channel_count, len(account.channel_keys), claim_count)
channel_count, len(account.certificates), claim_count)
class TestNetLedger(MainNetLedger):
@@ -4,20 +4,43 @@ import logging
from binascii import unhexlify
from datetime import datetime
from typing import Optional
from lbrynet.schema.schema import SECP256k1
from torba.client.basemanager import BaseWalletManager
from torba.rpc.jsonrpc import CodeMessageError
from lbrynet.wallet.ledger import MainNetLedger
from lbrynet.wallet.account import BaseAccount
from lbrynet.wallet.transaction import Transaction
from lbrynet.wallet.database import WalletDatabase
from lbrynet.wallet.dewies import dewies_to_lbc
from lbrynet.schema.claim import ClaimDict
from lbrynet.extras.wallet.ledger import MainNetLedger
from lbrynet.extras.wallet.account import BaseAccount, generate_certificate
from lbrynet.extras.wallet.transaction import Transaction
from lbrynet.extras.wallet.database import WalletDatabase
from lbrynet.extras.wallet.dewies import dewies_to_lbc
log = logging.getLogger(__name__)
class ReservedPoints:
def __init__(self, identifier, amount):
self.identifier = identifier
self.amount = amount
class BackwardsCompatibleNetwork:
def __init__(self, manager):
self.manager = manager
def get_local_height(self):
for ledger in self.manager.ledgers.values():
assert isinstance(ledger, MainNetLedger)
return ledger.headers.height
def get_server_height(self):
return self.get_local_height()
class LbryWalletManager(BaseWalletManager):
@property
@@ -28,10 +51,22 @@ class LbryWalletManager(BaseWalletManager):
def db(self) -> WalletDatabase:
return self.ledger.db
@property
def wallet(self):
return self
@property
def network(self):
return BackwardsCompatibleNetwork(self)
@property
def use_encryption(self):
return self.default_account.serialize_encrypted
@property
def is_first_run(self):
return True
@property
def is_wallet_unlocked(self):
return not self.default_account.encrypted
@@ -122,7 +157,7 @@ class LbryWalletManager(BaseWalletManager):
return receiving_addresses, change_addresses
@classmethod
async def from_lbrynet_config(cls, settings):
async def from_lbrynet_config(cls, settings, db):
ledger_id = {
'lbrycrd_main': 'lbc_mainnet',
@@ -187,12 +222,64 @@ class LbryWalletManager(BaseWalletManager):
def get_unused_address(self):
return self.default_account.receiving.get_or_create_usable_address()
def get_new_address(self):
return self.get_unused_address()
def reserve_points(self, address, amount: int):
# TODO: check if we have enough to cover amount
return ReservedPoints(address, amount)
async def send_amount_to_address(self, amount: int, destination_address: bytes, account=None):
account = account or self.default_account
tx = await Transaction.pay(amount, destination_address, [account], account)
await account.ledger.broadcast(tx)
return tx
async def send_claim_to_address(self, claim_id: str, destination_address: str, amount: Optional[int],
account=None):
account = account or self.default_account
claims = await account.get_claims(
claim_name_type__any={'is_claim': 1, 'is_update': 1}, # exclude is_supports
claim_id=claim_id
)
if not claims:
raise NameError(f"Claim not found: {claim_id}")
if not amount:
amount = claims[0].get_estimator(self.ledger).effective_amount
tx = await Transaction.update(
claims[0], ClaimDict.deserialize(claims[0].script.values['claim']), amount,
destination_address.encode(), [account], account
)
await self.ledger.broadcast(tx)
return tx
def send_points_to_address(self, reserved: ReservedPoints, amount: int, account=None):
destination_address: bytes = reserved.identifier.encode('latin1')
return self.send_amount_to_address(amount, destination_address, account)
async def resolve(self, *uris, **kwargs):
page = kwargs.get('page', 0)
page_size = kwargs.get('page_size', 10)
ledger: MainNetLedger = self.default_account.ledger
results = await ledger.resolve(page, page_size, *uris)
if 'error' not in results:
await self.old_db.save_claims_for_resolve([
value for value in results.values() if 'error' not in value
])
return results
async def get_claims_for_name(self, name: str):
response = await self.ledger.network.get_claims_for_name(name)
resolutions = await self.resolve(*(f"{claim['name']}#{claim['claim_id']}" for claim in response['claims']))
response['claims'] = [value.get('claim', value.get('certificate')) for value in resolutions.values()]
return response
async def address_is_mine(self, unknown_address, account):
match = await self.ledger.db.get_address(address=unknown_address, account=account)
if match is not None:
return True
return False
async def get_transaction(self, txid):
tx = await self.db.get_transaction(txid=txid)
if not tx:
@@ -304,10 +391,138 @@ class LbryWalletManager(BaseWalletManager):
history.append(item)
return history
@staticmethod
def get_utxos(account: BaseAccount):
return account.get_utxos()
async def claim_name(self, account, name, amount, claim_dict, certificate=None, claim_address=None):
claim = ClaimDict.load_dict(claim_dict)
if not claim_address:
claim_address = await account.receiving.get_or_create_usable_address()
if certificate:
claim = claim.sign(
certificate.private_key, claim_address, certificate.claim_id, curve=SECP256k1, name=name,
force_detached=False # TODO: delete it and make True default everywhere when its out
)
existing_claims = await account.get_claims(
claim_name_type__any={'is_claim': 1, 'is_update': 1}, # exclude is_supports
claim_name=name
)
if len(existing_claims) == 0:
tx = await Transaction.claim(
name, claim, amount, claim_address, [account], account
)
elif len(existing_claims) == 1:
tx = await Transaction.update(
existing_claims[0], claim, amount, claim_address, [account], account
)
else:
raise NameError(f"More than one other claim exists with the name '{name}'.")
await account.ledger.broadcast(tx)
await self.old_db.save_claims([self._old_get_temp_claim_info(
tx, tx.outputs[0], claim_address, claim_dict, name, dewies_to_lbc(amount)
)])
# TODO: release reserved tx outputs in case anything fails by this point
return tx
async def support_claim(self, claim_name, claim_id, amount, account):
holding_address = await account.receiving.get_or_create_usable_address()
tx = await Transaction.support(claim_name, claim_id, amount, holding_address, [account], account)
await account.ledger.broadcast(tx)
await self.old_db.save_supports(claim_id, [{
'txid': tx.id,
'nout': tx.position,
'address': holding_address,
'claim_id': claim_id,
'amount': dewies_to_lbc(amount)
}])
return tx
async def tip_claim(self, amount, claim_id, account):
claim_to_tip = await self.get_claim_by_claim_id(claim_id)
tx = await Transaction.support(
claim_to_tip['name'], claim_id, amount, claim_to_tip['address'], [account], account
)
await account.ledger.broadcast(tx)
await self.old_db.save_supports(claim_id, [{
'txid': tx.id,
'nout': tx.position,
'address': claim_to_tip['address'],
'claim_id': claim_id,
'amount': dewies_to_lbc(amount)
}])
return tx
async def abandon_claim(self, claim_id, txid, nout, account):
claim = await account.get_claim(claim_id=claim_id, txid=txid, nout=nout)
if not claim:
raise Exception('No claim found for the specified claim_id or txid:nout')
tx = await Transaction.abandon(claim, [account], account)
await account.ledger.broadcast(tx)
# TODO: release reserved tx outputs in case anything fails by this point
return tx
async def claim_new_channel(self, channel_name, amount, account):
address = await account.receiving.get_or_create_usable_address()
cert, key = generate_certificate()
tx = await Transaction.claim(channel_name, cert, amount, address, [account], account)
await account.ledger.broadcast(tx)
account.add_certificate_private_key(tx.outputs[0].ref, key.decode())
# TODO: release reserved tx outputs in case anything fails by this point
await self.old_db.save_claims([self._old_get_temp_claim_info(
tx, tx.outputs[0], address, cert, channel_name, dewies_to_lbc(amount)
)])
return tx
def _old_get_temp_claim_info(self, tx, txo, address, claim_dict, name, bid):
return {
"claim_id": txo.claim_id,
"name": name,
"amount": bid,
"address": address,
"txid": tx.id,
"nout": txo.position,
"value": claim_dict,
"height": -1,
"claim_sequence": -1,
}
def get_certificates(self, private_key_accounts, exclude_without_key=True, **constraints):
return self.db.get_certificates(
private_key_accounts=private_key_accounts,
exclude_without_key=exclude_without_key,
**constraints
)
def update_peer_address(self, peer, address):
pass # TODO: Data payments is disabled
def get_unused_address_for_peer(self, peer):
# TODO: Data payments is disabled
return self.get_unused_address()
def add_expected_payment(self, peer, amount):
pass # TODO: Data payments is disabled
def send_points(self, reserved_points, amount):
pass # TODO: Data payments is disabled
def cancel_point_reservation(self, reserved_points):
pass # fixme: disabled for now.
def save(self):
for wallet in self.wallets:
wallet.save()
def get_block(self, block_hash=None, height=None):
if height is None:
height = self.ledger.headers.height
if block_hash is None:
block_hash = self.ledger.headers.hash(height).decode()
return self.ledger.network.get_block(block_hash)
def get_claim_by_claim_id(self, claim_id):
return self.ledger.get_claim_by_claim_id(claim_id)
@@ -3,6 +3,9 @@ from torba.client.basenetwork import BaseNetwork
class Network(BaseNetwork):
def get_block(self, block_hash):
return self.rpc('blockchain.block.get_block', block_hash)
def get_server_height(self):
return self.rpc('blockchain.block.get_server_height')
+499
View File
@@ -0,0 +1,499 @@
import logging
from ecdsa import BadSignatureError
from binascii import unhexlify, hexlify
from lbrynet.extras.wallet.dewies import dewies_to_lbc
from lbrynet.error import UnknownNameError, UnknownClaimID, UnknownURI, UnknownOutpoint
from lbrynet.schema.address import is_address
from lbrynet.schema.claim import ClaimDict
from lbrynet.schema.decode import smart_decode
from lbrynet.schema.error import DecodeError
from lbrynet.schema.uri import parse_lbry_uri
from lbrynet.extras.wallet.claim_proofs import verify_proof, InvalidProofError
log = logging.getLogger(__name__)
class Resolver:
def __init__(self, claim_trie_root, height, transaction_class, hash160_to_address, network):
self.claim_trie_root = claim_trie_root
self.height = height
self.transaction_class = transaction_class
self.hash160_to_address = hash160_to_address
self.network = network
async def _handle_resolutions(self, resolutions, requested_uris, page, page_size):
results = {}
for uri in requested_uris:
resolution = (resolutions or {}).get(uri, {})
if resolution:
try:
results[uri] = _handle_claim_result(
await self._handle_resolve_uri_response(uri, resolution, page, page_size)
)
except (UnknownNameError, UnknownClaimID, UnknownURI) as err:
results[uri] = {'error': str(err)}
else:
results[uri] = {'error': "URI lbry://{} cannot be resolved".format(uri.replace("lbry://", ""))}
return results
async def _handle_resolve_uri_response(self, uri, resolution, page=0, page_size=10, raw=False):
result = {}
claim_trie_root = self.claim_trie_root
parsed_uri = parse_lbry_uri(uri)
certificate = None
# parse an included certificate
if 'certificate' in resolution:
certificate_response = resolution['certificate']['result']
certificate_resolution_type = resolution['certificate']['resolution_type']
if certificate_resolution_type == "winning" and certificate_response:
if 'height' in certificate_response:
height = certificate_response['height']
depth = self.height - height
certificate_result = _verify_proof(parsed_uri.name,
claim_trie_root,
certificate_response,
height, depth,
transaction_class=self.transaction_class,
hash160_to_address=self.hash160_to_address)
result['certificate'] = await self.parse_and_validate_claim_result(certificate_result,
raw=raw)
elif certificate_resolution_type == "claim_id":
result['certificate'] = await self.parse_and_validate_claim_result(certificate_response,
raw=raw)
elif certificate_resolution_type == "sequence":
result['certificate'] = await self.parse_and_validate_claim_result(certificate_response,
raw=raw)
else:
log.error("unknown response type: %s", certificate_resolution_type)
if 'certificate' in result:
certificate = result['certificate']
if 'unverified_claims_in_channel' in resolution:
max_results = len(resolution['unverified_claims_in_channel'])
result['claims_in_channel'] = max_results
else:
result['claims_in_channel'] = 0
else:
result['error'] = "claim not found"
result['success'] = False
result['uri'] = str(parsed_uri)
else:
certificate = None
# if this was a resolution for a name, parse the result
if 'claim' in resolution:
claim_response = resolution['claim']['result']
claim_resolution_type = resolution['claim']['resolution_type']
if claim_resolution_type == "winning" and claim_response:
if 'height' in claim_response:
height = claim_response['height']
depth = self.height - height
claim_result = _verify_proof(parsed_uri.name,
claim_trie_root,
claim_response,
height, depth,
transaction_class=self.transaction_class,
hash160_to_address=self.hash160_to_address)
result['claim'] = await self.parse_and_validate_claim_result(claim_result,
certificate,
raw)
elif claim_resolution_type == "claim_id":
result['claim'] = await self.parse_and_validate_claim_result(claim_response,
certificate,
raw)
elif claim_resolution_type == "sequence":
result['claim'] = await self.parse_and_validate_claim_result(claim_response,
certificate,
raw)
else:
log.error("unknown response type: %s", claim_resolution_type)
# if this was a resolution for a name in a channel make sure there is only one valid
# match
elif 'unverified_claims_for_name' in resolution and 'certificate' in result:
unverified_claims_for_name = resolution['unverified_claims_for_name']
channel_info = await self.get_channel_claims_page(unverified_claims_for_name,
result['certificate'], page=1)
claims_in_channel, upper_bound = channel_info
if not claims_in_channel:
log.error("No valid claims for this name for this channel")
elif len(claims_in_channel) > 1:
log.warning("Multiple signed claims for the same name.")
winner = pick_winner_from_channel_path_collision(claims_in_channel)
if winner:
result['claim'] = winner
else:
log.error("No valid claims for this name for this channel")
else:
result['claim'] = claims_in_channel[0]
# parse and validate claims in a channel iteratively into pages of results
elif 'unverified_claims_in_channel' in resolution and 'certificate' in result:
ids_to_check = resolution['unverified_claims_in_channel']
channel_info = await self.get_channel_claims_page(ids_to_check, result['certificate'],
page=page, page_size=page_size)
claims_in_channel, upper_bound = channel_info
if claims_in_channel:
result['claims_in_channel'] = claims_in_channel
elif 'error' not in result:
return {'error': 'claim not found', 'success': False, 'uri': str(parsed_uri)}
# invalid signatures can only return outside a channel
if result.get('claim', {}).get('has_signature', False):
if parsed_uri.path and not result['claim']['signature_is_valid']:
return {'error': 'claim not found', 'success': False, 'uri': str(parsed_uri)}
return result
async def get_certificate_and_validate_result(self, claim_result):
if not claim_result or 'value' not in claim_result:
return claim_result
certificate = None
certificate_id = smart_decode(claim_result['value']).certificate_id
if certificate_id:
certificate = await self.network.get_claims_by_ids(certificate_id)
certificate = certificate.pop(certificate_id) if certificate else None
return await self.parse_and_validate_claim_result(claim_result, certificate=certificate)
async def parse_and_validate_claim_result(self, claim_result, certificate=None, raw=False):
if not claim_result or 'value' not in claim_result:
return claim_result
claim_result['decoded_claim'] = False
decoded = None
if not raw:
claim_value = claim_result['value']
try:
decoded = smart_decode(claim_value)
claim_result['value'] = decoded.claim_dict
claim_result['decoded_claim'] = True
except DecodeError:
pass
if decoded:
claim_result['has_signature'] = False
if decoded.has_signature:
if certificate is None:
log.info("fetching certificate to check claim signature")
certificate = await self.network.get_claims_by_ids(decoded.certificate_id)
if not certificate:
log.warning('Certificate %s not found', decoded.certificate_id)
claim_result['has_signature'] = True
claim_result['signature_is_valid'] = False
validated, channel_name = validate_claim_signature_and_get_channel_name(
decoded, certificate, claim_result['address'], claim_result['name'])
claim_result['channel_name'] = channel_name
if validated:
claim_result['signature_is_valid'] = True
if 'height' in claim_result and claim_result['height'] is None:
claim_result['height'] = -1
if 'amount' in claim_result:
claim_result = format_amount_value(claim_result)
claim_result['permanent_url'] = _get_permanent_url(claim_result, decoded.certificate_id if decoded else None)
return claim_result
@staticmethod
def prepare_claim_queries(start_position, query_size, channel_claim_infos):
queries = [tuple()]
names = {}
# a table of index counts for the sorted claim ids, including ignored claims
absolute_position_index = {}
block_sorted_infos = sorted(channel_claim_infos.items(), key=lambda x: int(x[1][1]))
per_block_infos = {}
for claim_id, (name, height) in block_sorted_infos:
claims = per_block_infos.get(height, [])
claims.append((claim_id, name))
per_block_infos[height] = sorted(claims, key=lambda x: int(x[0], 16))
abs_position = 0
for height in sorted(per_block_infos.keys(), reverse=True):
for claim_id, name in per_block_infos[height]:
names[claim_id] = name
absolute_position_index[claim_id] = abs_position
if abs_position >= start_position:
if len(queries[-1]) >= query_size:
queries.append(tuple())
queries[-1] += (claim_id,)
abs_position += 1
return queries, names, absolute_position_index
async def iter_channel_claims_pages(self, queries, claim_positions, claim_names, certificate,
page_size=10):
# lbryum server returns a dict of {claim_id: (name, claim_height)}
# first, sort the claims by block height (and by claim id int value within a block).
# map the sorted claims into getclaimsbyids queries of query_size claim ids each
# send the batched queries to lbryum server and iteratively validate and parse
# the results, yield a page of results at a time.
# these results can include those where `signature_is_valid` is False. if they are skipped,
# page indexing becomes tricky, as the number of results isn't known until after having
# processed them.
# TODO: fix ^ in lbrynet.schema
async def iter_validate_channel_claims():
formatted_claims = []
for claim_ids in queries:
batch_result = await self.network.get_claims_by_ids(*claim_ids)
for claim_id in claim_ids:
claim = batch_result[claim_id]
if claim['name'] == claim_names[claim_id]:
formatted_claim = await self.parse_and_validate_claim_result(claim, certificate)
formatted_claim['absolute_channel_position'] = claim_positions[
claim['claim_id']]
formatted_claims.append(formatted_claim)
else:
log.warning("ignoring claim with name mismatch %s %s", claim['name'],
claim['claim_id'])
return formatted_claims
results = []
for claim in await iter_validate_channel_claims():
results.append(claim)
# if there is a full page of results, yield it
if len(results) and len(results) % page_size == 0:
return results[-page_size:]
return results
async def get_channel_claims_page(self, channel_claim_infos, certificate, page, page_size=10):
page = page or 0
page_size = max(page_size, 1)
if page_size > 500:
raise Exception("page size above maximum allowed")
start_position = (page - 1) * page_size
queries, names, claim_positions = self.prepare_claim_queries(start_position, page_size,
channel_claim_infos)
page_generator = await self.iter_channel_claims_pages(queries, claim_positions, names,
certificate, page_size=page_size)
upper_bound = len(claim_positions)
if not page:
return None, upper_bound
if start_position > upper_bound:
raise IndexError("claim %i greater than max %i" % (start_position, upper_bound))
return page_generator, upper_bound
# Format amount to be decimal encoded string
# Format value to be hex encoded string
# TODO: refactor. Came from lbryum, there could be another part of torba doing it
def format_amount_value(obj):
if isinstance(obj, dict):
for k, v in obj.items():
if k in ('amount', 'effective_amount'):
if not isinstance(obj[k], float):
obj[k] = dewies_to_lbc(obj[k])
elif k == 'supports' and isinstance(v, list):
obj[k] = [{'txid': txid, 'nout': nout, 'amount': dewies_to_lbc(amount)}
for (txid, nout, amount) in v]
elif isinstance(v, (list, dict)):
obj[k] = format_amount_value(v)
elif isinstance(obj, list):
obj = [format_amount_value(o) for o in obj]
return obj
def _get_permanent_url(claim_result, certificate_id):
if certificate_id:
return f"{claim_result['channel_name']}#{certificate_id}/{claim_result['name']}"
else:
return f"{claim_result['name']}#{claim_result['claim_id']}"
def _verify_proof(name, claim_trie_root, result, height, depth, transaction_class, hash160_to_address):
"""
Verify proof for name claim
"""
def _build_response(name, value, claim_id, txid, n, amount, effective_amount,
claim_sequence, claim_address, supports):
r = {
'name': name,
'value': hexlify(value),
'claim_id': claim_id,
'txid': txid,
'nout': n,
'amount': amount,
'effective_amount': effective_amount,
'height': height,
'depth': depth,
'claim_sequence': claim_sequence,
'address': claim_address,
'supports': supports
}
return r
def _parse_proof_result(name, result):
support_amount = sum([amt for (stxid, snout, amt) in result['supports']])
supports = result['supports']
if 'txhash' in result['proof'] and 'nOut' in result['proof']:
if 'transaction' in result:
tx = transaction_class(raw=unhexlify(result['transaction']))
nOut = result['proof']['nOut']
if result['proof']['txhash'] == tx.id:
if 0 <= nOut < len(tx.outputs):
claim_output = tx.outputs[nOut]
effective_amount = claim_output.amount + support_amount
claim_address = hash160_to_address(claim_output.script.values['pubkey_hash'])
claim_id = result['claim_id']
claim_sequence = result['claim_sequence']
claim_script = claim_output.script
decoded_name = claim_script.values['claim_name'].decode()
decoded_value = claim_script.values['claim']
if decoded_name == name:
return _build_response(name, decoded_value, claim_id,
tx.id, nOut, claim_output.amount,
effective_amount, claim_sequence,
claim_address, supports)
return {'error': 'name in proof did not match requested name'}
outputs = len(tx['outputs'])
return {'error': 'invalid nOut: %d (let(outputs): %d' % (nOut, outputs)}
return {'error': "computed txid did not match given transaction: %s vs %s" %
(tx.id, result['proof']['txhash'])
}
return {'error': "didn't receive a transaction with the proof"}
return {'error': 'name is not claimed'}
if 'proof' in result:
proof_name = name
if 'name' in result:
proof_name = result['name']
name = result['name']
if 'normalized_name' in result:
proof_name = result['normalized_name']
try:
verify_proof(result['proof'], claim_trie_root, proof_name)
except InvalidProofError:
return {'error': "Proof was invalid"}
return _parse_proof_result(name, result)
else:
return {'error': "proof not in result"}
def validate_claim_signature_and_get_channel_name(claim, certificate_claim,
claim_address, name, decoded_certificate=None):
if not certificate_claim:
return False, None
if 'value' not in certificate_claim:
log.warning('Got an invalid claim while parsing certificates, please report: %s', certificate_claim)
return False, None
certificate = decoded_certificate or smart_decode(certificate_claim['value'])
if not isinstance(certificate, ClaimDict):
raise TypeError("Certificate is not a ClaimDict: %s" % str(type(certificate)))
if _validate_signed_claim(claim, claim_address, name, certificate):
return True, certificate_claim['name']
return False, None
def _validate_signed_claim(claim, claim_address, name, certificate):
if not claim.has_signature:
raise Exception("Claim is not signed")
if not is_address(claim_address):
raise Exception("Not given a valid claim address")
try:
if claim.validate_signature(claim_address, certificate.protobuf, name):
return True
except BadSignatureError:
# print_msg("Signature for %s is invalid" % claim_id)
return False
except Exception as err:
log.error("Signature for %s is invalid, reason: %s - %s", claim_address,
str(type(err)), err)
return False
return False
# TODO: The following came from code handling lbryum results. Now that it's all in one place a refactor should unify it.
def _decode_claim_result(claim):
if 'has_signature' in claim and claim['has_signature']:
if not claim['signature_is_valid']:
log.warning("lbry://%s#%s has an invalid signature",
claim['name'], claim['claim_id'])
if 'value' not in claim:
log.warning('Got an invalid claim while parsing, please report: %s', claim)
claim['hex'] = None
claim['value'] = None
backend_message = ' SDK message: ' + claim['error'] if 'error' in claim else ''
claim['error'] = "Failed to parse: missing value." + backend_message
return claim
try:
decoded = smart_decode(claim['value'])
claim_dict = decoded.claim_dict
claim['value'] = claim_dict
claim['hex'] = hexlify(decoded.serialized)
except DecodeError:
claim['hex'] = claim['value']
claim['value'] = None
claim['error'] = "Failed to decode value"
return claim
def _handle_claim_result(results):
if not results:
#TODO: cannot determine what name we searched for here
# we should fix lbryum commands that return None
raise UnknownNameError("")
if 'error' in results:
if results['error'] in ['name is not claimed', 'claim not found']:
if 'claim_id' in results:
raise UnknownClaimID(results['claim_id'])
if 'name' in results:
raise UnknownNameError(results['name'])
if 'uri' in results:
raise UnknownURI(results['uri'])
if 'outpoint' in results:
raise UnknownOutpoint(results['outpoint'])
raise Exception(results['error'])
# case where return value is {'certificate':{'txid', 'value',...},...}
if 'certificate' in results:
results['certificate'] = _decode_claim_result(results['certificate'])
# case where return value is {'claim':{'txid','value',...},...}
if 'claim' in results:
results['claim'] = _decode_claim_result(results['claim'])
# case where return value is {'txid','value',...}
# returned by queries that are not name resolve related
# (getclaimbyoutpoint, getclaimbyid, getclaimsfromtx)
elif 'value' in results:
results = _decode_claim_result(results)
# case where there is no 'certificate', 'value', or 'claim' key
elif 'certificate' not in results:
msg = f'result in unexpected format:{results}'
assert False, msg
return results
def pick_winner_from_channel_path_collision(claims_in_channel):
# we should be doing this by effective amount so we pick the controlling claim, however changing the resolved
# claim triggers another issue where 2 claims cant be saved for the same file. This code picks the oldest, so it
# stays the same. Using effective amount would change the resolved claims for a channel path on takeovers,
# potentially triggering that.
winner = None
for claim in claims_in_channel:
if not claim['signature_is_valid']:
continue
if winner is None:
winner = claim
elif claim['height'] < winner['height'] or \
(claim['height'] == winner['height'] and claim['nout'] < winner['nout']):
winner = claim
return winner
File renamed without changes.
File renamed without changes.
@@ -1,14 +1,15 @@
import hashlib
import struct
from binascii import unhexlify
import msgpack
from lbrynet.wallet.transaction import Transaction, Output
from torba.server.hash import hash_to_hex_str
from torba.server.block_processor import BlockProcessor
from lbrynet.schema.claim import Claim
from lbrynet.schema.uri import parse_lbry_uri
from lbrynet.schema.decode import smart_decode
from lbrynet.wallet.server.model import ClaimInfo
from lbrynet.extras.wallet.server.model import NameClaim, ClaimInfo, ClaimUpdate, ClaimSupport
class LBRYBlockProcessor(BlockProcessor):
@@ -36,31 +37,32 @@ class LBRYBlockProcessor(BlockProcessor):
undo_info = []
add_undo = undo_info.append
update_inputs = set()
for etx, txid in txs:
for tx, txid in txs:
update_inputs.clear()
tx = Transaction(etx.serialize())
for index, output in enumerate(tx.outputs):
if not output.is_claim:
continue
if output.script.is_claim_name:
add_undo(self.advance_claim_name_transaction(output, height, txid, index))
elif output.script.is_update_claim:
update_input = self.db.get_update_input(output.claim_hash, tx.inputs)
if update_input:
update_inputs.add(update_input)
add_undo(self.advance_update_claim(output, height, txid, index))
else:
info = (hash_to_hex_str(txid), output.claim_id,)
self.logger.error("REJECTED: {} updating {}".format(*info))
if tx.has_claims:
for index, output in enumerate(tx.outputs):
claim = output.claim
if isinstance(claim, NameClaim):
add_undo(self.advance_claim_name_transaction(output, height, txid, index))
elif isinstance(claim, ClaimUpdate):
update_input = self.db.get_update_input(claim, tx.inputs)
if update_input:
update_inputs.add(update_input)
add_undo(self.advance_update_claim(output, height, txid, index))
else:
info = (hash_to_hex_str(txid), hash_to_hex_str(claim.claim_id),)
self.logger.error("REJECTED: {} updating {}".format(*info))
elif isinstance(claim, ClaimSupport):
self.advance_support(claim, txid, index, height, output.value)
for txin in tx.inputs:
if txin not in update_inputs:
abandoned_claim_id = self.db.abandon_spent(txin.txo_ref.tx_ref.hash, txin.txo_ref.position)
abandoned_claim_id = self.db.abandon_spent(txin.prev_hash, txin.prev_idx)
if abandoned_claim_id:
add_undo((abandoned_claim_id, self.db.get_claim_info(abandoned_claim_id)))
return undo_info
def advance_update_claim(self, output: Output, height, txid, nout):
claim_id = output.claim_hash
def advance_update_claim(self, output, height, txid, nout):
claim_id = output.claim.claim_id
claim_info = self.claim_info_from_output(output, txid, nout, height)
old_claim_info = self.db.get_claim_info(claim_id)
self.db.put_claim_id_for_outpoint(old_claim_info.txid, old_claim_info.nout, None)
@@ -72,8 +74,8 @@ class LBRYBlockProcessor(BlockProcessor):
self.db.put_claim_id_for_outpoint(txid, nout, claim_id)
return claim_id, old_claim_info
def advance_claim_name_transaction(self, output: Output, height, txid, nout):
claim_id = output.claim_hash
def advance_claim_name_transaction(self, output, height, txid, nout):
claim_id = claim_id_hash(txid, nout)
claim_info = self.claim_info_from_output(output, txid, nout, height)
if claim_info.cert_id:
self.db.put_claim_id_signed_by_cert_id(claim_info.cert_id, claim_id)
@@ -111,7 +113,7 @@ class LBRYBlockProcessor(BlockProcessor):
if undo_claim_info:
self.db.put_claim_info(claim_id, undo_claim_info)
if undo_claim_info.cert_id:
cert_id = self._checksig(undo_claim_info.value, undo_claim_info.address)
cert_id = self._checksig(undo_claim_info.name, undo_claim_info.value, undo_claim_info.address)
self.db.put_claim_id_signed_by_cert_id(cert_id, claim_id)
self.db.put_claim_id_for_outpoint(undo_claim_info.txid, undo_claim_info.nout, claim_id)
@@ -131,24 +133,37 @@ class LBRYBlockProcessor(BlockProcessor):
self.db.batched_flush_claims()
return await super().flush(flush_utxos)
def claim_info_from_output(self, output: Output, txid, nout, height):
address = self.coin.address_from_script(output.script.source)
name, value, cert_id = output.script.values['claim_name'], output.script.values['claim'], None
assert txid and address
cert_id = self._checksig(value, address)
return ClaimInfo(name, value, txid, nout, output.amount, address, height, cert_id)
def advance_support(self, claim_support, txid, nout, height, amount):
# TODO: check for more controller claim rules, like takeover or ordering
pass
def _checksig(self, value, address):
def claim_info_from_output(self, output, txid, nout, height):
amount = output.value
address = self.coin.address_from_script(output.pk_script)
name, value, cert_id = output.claim.name, output.claim.value, None
assert txid and address
cert_id = self._checksig(name, value, address)
return ClaimInfo(name, value, txid, nout, amount, address, height, cert_id)
def _checksig(self, name, value, address):
try:
claim_dict = Claim.from_bytes(value)
cert_id = claim_dict.signing_channel_hash
parse_lbry_uri(name.decode()) # skip invalid names
claim_dict = smart_decode(value)
cert_id = unhexlify(claim_dict.certificate_id)[::-1]
if not self.should_validate_signatures:
return cert_id
if cert_id:
cert_claim = self.db.get_claim_info(cert_id)
if cert_claim:
certificate = Claim.from_bytes(cert_claim.value)
certificate = smart_decode(cert_claim.value)
claim_dict.validate_signature(address, certificate)
return cert_id
except Exception:
except Exception as e:
pass
def claim_id_hash(txid, n):
# TODO: This should be in lbryschema
packed = txid + struct.pack('>I', n)
md = hashlib.new('ripemd160')
md.update(hashlib.sha256(packed).digest())
return md.digest()
@@ -1,29 +1,32 @@
import struct
from lbrynet.wallet.script import OutputScript
from torba.server.script import ScriptPubKey, OpCodes
from torba.server.script import ScriptPubKey, _match_ops, OpCodes
from torba.server.util import cachedproperty
from torba.server.hash import hash_to_hex_str, HASHX_LEN
from hashlib import sha256
from torba.server.coins import Coin, CoinError
from lbrynet.extras.wallet.server.opcodes import decode_claim_script, opcodes as lbry_opcodes
class LBC(Coin):
from .session import LBRYElectrumX
from .block_processor import LBRYBlockProcessor
from .tx import LBRYDeserializer
from .daemon import LBCDaemon
from .db import LBRYDB
DAEMON = LBCDaemon
SESSIONCLS = LBRYElectrumX
BLOCK_PROCESSOR = LBRYBlockProcessor
DB = LBRYDB
DESERIALIZER = LBRYDeserializer
NAME = "LBRY"
SHORTNAME = "LBC"
NET = "mainnet"
BASIC_HEADER_SIZE = 112
CHUNK_SIZE = 96
XPUB_VERBYTES = bytes.fromhex("0488b21e")
XPRV_VERBYTES = bytes.fromhex("0488ade4")
XPUB_VERBYTES = bytes.fromhex("019C354f")
XPRV_VERBYTES = bytes.fromhex("019C3118")
P2PKH_VERBYTE = bytes.fromhex("55")
P2SH_VERBYTES = bytes.fromhex("7A")
WIF_BYTE = bytes.fromhex("1C")
@@ -85,14 +88,28 @@ class LBC(Coin):
def claim_address_handler(cls, script):
'''Parse a claim script, returns the address
'''
output = OutputScript(script)
if output.is_pay_pubkey_hash:
return cls.P2PKH_address_from_hash160(output.values['pubkey_hash'])
if output.is_pay_script_hash:
return cls.P2SH_address_from_hash160(output.values['script_hash'])
if output.is_pay_pubkey:
return cls.P2PKH_address_from_pubkey(output.values['pubkey'])
if output.is_return_data:
decoded = decode_claim_script(script)
if not decoded:
return None
ops = []
for op, data, _ in decoded[1]:
if not data:
ops.append(op)
else:
ops.append((op, data,))
match = _match_ops
TO_ADDRESS_OPS = [OpCodes.OP_DUP, OpCodes.OP_HASH160, -1,
OpCodes.OP_EQUALVERIFY, OpCodes.OP_CHECKSIG]
TO_P2SH_OPS = [OpCodes.OP_HASH160, -1, OpCodes.OP_EQUAL]
TO_PUBKEY_OPS = [-1, OpCodes.OP_CHECKSIG]
if match(ops, TO_ADDRESS_OPS):
return cls.P2PKH_address_from_hash160(ops[2][-1])
if match(ops, TO_P2SH_OPS):
return cls.P2SH_address_from_hash160(ops[1][-1])
if match(ops, TO_PUBKEY_OPS):
return cls.P2PKH_address_from_pubkey(ops[0][-1])
if ops and ops[0] == OpCodes.OP_RETURN:
return None
return None
@@ -104,9 +121,9 @@ class LBC(Coin):
if script and script[0] == OpCodes.OP_RETURN:
return None
if script[0] in [
OutputScript.OP_CLAIM_NAME,
OutputScript.OP_UPDATE_CLAIM,
OutputScript.OP_SUPPORT_CLAIM,
lbry_opcodes.OP_CLAIM_NAME,
lbry_opcodes.OP_SUPPORT_CLAIM,
lbry_opcodes.OP_UPDATE_CLAIM
]:
return cls.address_to_hashX(cls.claim_address_handler(script))
else:
File renamed without changes.
@@ -6,7 +6,7 @@ from torba.server.hash import hash_to_hex_str
from torba.server.db import DB
from lbrynet.wallet.server.model import ClaimInfo
from lbrynet.extras.wallet.server.model import ClaimInfo
class LBRYDB(DB):
@@ -161,12 +161,13 @@ class LBRYDB(DB):
self.logger.info("[+] Adding claim info for: {}".format(hash_to_hex_str(claim_id)))
self.claim_cache[claim_id] = claim_info.serialized
def get_update_input(self, claim_id, inputs):
def get_update_input(self, claim, inputs):
claim_id = claim.claim_id
claim_info = self.get_claim_info(claim_id)
if not claim_info:
return False
for input in inputs:
if (input.txo_ref.tx_ref.hash, input.txo_ref.position) == (claim_info.txid, claim_info.nout):
if input.prev_hash == claim_info.txid and input.prev_idx == claim_info.nout:
return input
return False
+47
View File
@@ -0,0 +1,47 @@
from collections import namedtuple
import msgpack
from torba.server.util import cachedproperty
# Classes representing data and their serializers, if any.
class ClaimInfo(namedtuple("NameClaim", "name value txid nout amount address height cert_id")):
'''Claim information as its stored on database'''
@classmethod
def from_serialized(cls, serialized):
return cls(*msgpack.loads(serialized))
@property
def serialized(self):
return msgpack.dumps(self)
class NameClaim(namedtuple("NameClaim", "name value")):
pass
class ClaimUpdate(namedtuple("ClaimUpdate", "name claim_id value")):
pass
class ClaimSupport(namedtuple("ClaimSupport", "name claim_id")):
pass
class LBRYTx(namedtuple("Tx", "version inputs outputs locktime")):
'''Transaction that can contain claim, update or support in its outputs.'''
@cachedproperty
def is_coinbase(self):
return self.inputs[0].is_coinbase
@cachedproperty
def has_claims(self):
for output in self.outputs:
if output.claim:
return True
return False
class TxClaimOutput(namedtuple("TxClaimOutput", "value pk_script claim")):
pass
+126
View File
@@ -0,0 +1,126 @@
import struct
from torba.server.enum import Enumeration
from lbrynet.extras.wallet.server.model import NameClaim, ClaimSupport, ClaimUpdate
# TODO: Take this to lbryschema (it's also on lbryum and lbryum-server)
opcodes = Enumeration("Opcodes", [
("OP_0", 0), ("OP_PUSHDATA1", 76), "OP_PUSHDATA2", "OP_PUSHDATA4", "OP_1NEGATE", "OP_RESERVED",
"OP_1", "OP_2", "OP_3", "OP_4", "OP_5", "OP_6", "OP_7",
"OP_8", "OP_9", "OP_10", "OP_11", "OP_12", "OP_13", "OP_14", "OP_15", "OP_16",
"OP_NOP", "OP_VER", "OP_IF", "OP_NOTIF", "OP_VERIF", "OP_VERNOTIF", "OP_ELSE", "OP_ENDIF",
"OP_VERIFY",
"OP_RETURN", "OP_TOALTSTACK", "OP_FROMALTSTACK", "OP_2DROP", "OP_2DUP", "OP_3DUP", "OP_2OVER",
"OP_2ROT", "OP_2SWAP",
"OP_IFDUP", "OP_DEPTH", "OP_DROP", "OP_DUP", "OP_NIP", "OP_OVER", "OP_PICK", "OP_ROLL",
"OP_ROT",
"OP_SWAP", "OP_TUCK", "OP_CAT", "OP_SUBSTR", "OP_LEFT", "OP_RIGHT", "OP_SIZE", "OP_INVERT",
"OP_AND",
"OP_OR", "OP_XOR", "OP_EQUAL", "OP_EQUALVERIFY", "OP_RESERVED1", "OP_RESERVED2", "OP_1ADD",
"OP_1SUB", "OP_2MUL",
"OP_2DIV", "OP_NEGATE", "OP_ABS", "OP_NOT", "OP_0NOTEQUAL", "OP_ADD", "OP_SUB", "OP_MUL",
"OP_DIV",
"OP_MOD", "OP_LSHIFT", "OP_RSHIFT", "OP_BOOLAND", "OP_BOOLOR",
"OP_NUMEQUAL", "OP_NUMEQUALVERIFY", "OP_NUMNOTEQUAL", "OP_LESSTHAN",
"OP_GREATERTHAN", "OP_LESSTHANOREQUAL", "OP_GREATERTHANOREQUAL", "OP_MIN", "OP_MAX",
"OP_WITHIN", "OP_RIPEMD160", "OP_SHA1", "OP_SHA256", "OP_HASH160",
"OP_HASH256", "OP_CODESEPARATOR", "OP_CHECKSIG", "OP_CHECKSIGVERIFY", "OP_CHECKMULTISIG",
"OP_CHECKMULTISIGVERIFY", "OP_NOP1", "OP_NOP2", "OP_NOP3", "OP_NOP4", "OP_NOP5",
"OP_CLAIM_NAME",
"OP_SUPPORT_CLAIM", "OP_UPDATE_CLAIM",
("OP_SINGLEBYTE_END", 0xF0),
("OP_DOUBLEBYTE_BEGIN", 0xF000),
"OP_PUBKEY", "OP_PUBKEYHASH",
("OP_INVALIDOPCODE", 0xFFFF),
])
def script_GetOp(bytes):
i = 0
while i < len(bytes):
vch = None
opcode = bytes[i]
i += 1
if opcode <= opcodes.OP_PUSHDATA4:
nSize = opcode
if opcode == opcodes.OP_PUSHDATA1:
nSize = bytes[i]
i += 1
elif opcode == opcodes.OP_PUSHDATA2:
(nSize,) = struct.unpack_from('<H', bytes, i)
i += 2
elif opcode == opcodes.OP_PUSHDATA4:
(nSize,) = struct.unpack_from('<I', bytes, i)
i += 4
if i + nSize > len(bytes):
vch = "_INVALID_" + bytes[i:]
i = len(bytes)
else:
vch = bytes[i:i + nSize]
i += nSize
yield (opcode, vch, i)
def decode_claim_script(bytes_script):
try:
decoded_script = [x for x in script_GetOp(bytes_script)]
except Exception as e:
print(e)
return None
if len(decoded_script) <= 6:
return False
op = 0
claim_type = decoded_script[op][0]
if claim_type == opcodes.OP_UPDATE_CLAIM:
if len(decoded_script) <= 7:
return False
if claim_type not in [
opcodes.OP_CLAIM_NAME,
opcodes.OP_SUPPORT_CLAIM,
opcodes.OP_UPDATE_CLAIM
]:
return False
op += 1
value = None
claim_id = None
claim = None
if not 0 <= decoded_script[op][0] <= opcodes.OP_PUSHDATA4:
return False
name = decoded_script[op][1]
op += 1
if not 0 <= decoded_script[op][0] <= opcodes.OP_PUSHDATA4:
return False
if decoded_script[0][0] in [
opcodes.OP_SUPPORT_CLAIM,
opcodes.OP_UPDATE_CLAIM
]:
claim_id = decoded_script[op][1]
if len(claim_id) != 20:
return False
else:
value = decoded_script[op][1]
op += 1
if decoded_script[0][0] == opcodes.OP_UPDATE_CLAIM:
value = decoded_script[op][1]
op += 1
if decoded_script[op][0] != opcodes.OP_2DROP:
return False
op += 1
if decoded_script[op][0] != opcodes.OP_DROP and decoded_script[0][0] == opcodes.OP_CLAIM_NAME:
return False
elif decoded_script[op][0] != opcodes.OP_2DROP and decoded_script[0][0] == opcodes.OP_UPDATE_CLAIM:
return False
op += 1
if decoded_script[0][0] == opcodes.OP_CLAIM_NAME:
if name is None or value is None:
return False
claim = NameClaim(name, value)
elif decoded_script[0][0] == opcodes.OP_UPDATE_CLAIM:
if name is None or value is None or claim_id is None:
return False
claim = ClaimUpdate(name, claim_id, value)
elif decoded_script[0][0] == opcodes.OP_SUPPORT_CLAIM:
if name is None or claim_id is None:
return False
claim = ClaimSupport(name, claim_id)
return claim, decoded_script[op:]
@@ -7,9 +7,10 @@ from torba.server.hash import hash_to_hex_str
from torba.server.session import ElectrumX
from torba.server import util
from lbrynet.schema.uri import parse_lbry_uri, CLAIM_ID_MAX_LENGTH, URIParseError
from lbrynet.wallet.server.block_processor import LBRYBlockProcessor
from lbrynet.wallet.server.db import LBRYDB
from lbrynet.schema.uri import parse_lbry_uri, CLAIM_ID_MAX_LENGTH
from lbrynet.schema.error import URIParseError
from lbrynet.extras.wallet.server.block_processor import LBRYBlockProcessor
from lbrynet.extras.wallet.server.db import LBRYDB
class LBRYElectrumX(ElectrumX):
@@ -41,6 +42,7 @@ class LBRYElectrumX(ElectrumX):
'blockchain.claimtrie.getvaluesforuris': self.claimtrie_getvalueforuris,
'blockchain.claimtrie.getclaimssignedbyid': self.claimtrie_getclaimssignedbyid,
'blockchain.block.get_server_height': self.get_server_height,
'blockchain.block.get_block': self.get_block,
}
# fixme: methods we use but shouldnt be using anymore. To be removed when torba goes out
handlers.update({
@@ -103,6 +105,9 @@ class LBRYElectrumX(ElectrumX):
verbose = False
return await self.daemon_request('getrawtransaction', tx_hash, verbose)
async def get_block(self, block_hash):
return await self.daemon.deserialised_block(block_hash)
async def get_server_height(self):
return self.bp.height
@@ -237,7 +242,7 @@ class LBRYElectrumX(ElectrumX):
"txid": claim['txid'],
"nout": claim['n'],
"amount": amount,
"depth": self.db.db_height - height + 1,
"depth": self.db.db_height - height,
"height": height,
"value": hexlify(claim['value'].encode('ISO-8859-1')).decode(),
"address": address, # from index
+21
View File
@@ -0,0 +1,21 @@
from torba.server.tx import Deserializer
from lbrynet.extras.wallet.server.opcodes import decode_claim_script
from lbrynet.extras.wallet.server.model import TxClaimOutput, LBRYTx
class LBRYDeserializer(Deserializer):
def _read_output(self):
value = self._read_le_int64()
script = self._read_varbytes() # pk_script
claim = decode_claim_script(script)
claim = claim[0] if claim else None
return TxClaimOutput(value, script, claim)
def read_tx(self):
return LBRYTx(
self._read_le_int32(), # version
self._read_inputs(), # inputs
self._read_outputs(), # outputs
self._read_le_uint32() # locktime
)
@@ -1,21 +1,13 @@
import struct
import hashlib
from binascii import hexlify, unhexlify
from typing import List, Optional
from typing import List, Iterable, Optional
import ecdsa
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.serialization import load_der_public_key
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives.asymmetric.utils import Prehashed
from ecdsa.util import sigencode_der
from torba.client.basetransaction import BaseTransaction, BaseInput, BaseOutput, ReadOnlyList
from torba.client.hash import hash160, sha256, Base58
from lbrynet.schema.claim import Claim
from lbrynet.wallet.account import Account
from lbrynet.wallet.script import InputScript, OutputScript
from torba.client.basetransaction import BaseTransaction, BaseInput, BaseOutput
from torba.client.hash import hash160
from lbrynet.schema.decode import smart_decode
from lbrynet.schema.claim import ClaimDict
from lbrynet.extras.wallet.account import Account
from lbrynet.extras.wallet.script import InputScript, OutputScript
class Input(BaseInput):
@@ -27,11 +19,12 @@ class Output(BaseOutput):
script: OutputScript
script_class = OutputScript
__slots__ = 'channel', 'private_key'
__slots__ = '_claim_dict', 'channel', 'private_key'
def __init__(self, *args, channel: Optional['Output'] = None,
private_key: Optional[str] = None, **kwargs) -> None:
super().__init__(*args, **kwargs)
self._claim_dict = None
self.channel = channel
self.private_key = private_key
@@ -51,17 +44,14 @@ class Output(BaseOutput):
return self.script.is_claim_name or self.script.is_update_claim
@property
def claim_hash(self) -> bytes:
def claim_id(self) -> str:
if self.script.is_claim_name:
return hash160(self.tx_ref.hash + struct.pack('>I', self.position))
claim_id = hash160(self.tx_ref.hash + struct.pack('>I', self.position))
elif self.script.is_update_claim or self.script.is_support_claim:
return self.script.values['claim_id']
claim_id = self.script.values['claim_id']
else:
raise ValueError('No claim_id associated.')
@property
def claim_id(self) -> str:
return hexlify(self.claim_hash[::-1]).decode()
return hexlify(claim_id[::-1]).decode()
@property
def claim_name(self) -> str:
@@ -70,92 +60,38 @@ class Output(BaseOutput):
raise ValueError('No claim_name associated.')
@property
def claim(self) -> Claim:
def claim(self) -> ClaimDict:
if self.is_claim:
if not isinstance(self.script.values['claim'], Claim):
self.script.values['claim'] = Claim.from_bytes(self.script.values['claim'])
return self.script.values['claim']
return smart_decode(self.script.values['claim'])
raise ValueError('Only claim name and claim update have the claim payload.')
@property
def claim_dict(self) -> dict:
if self._claim_dict is None:
self._claim_dict = self.claim.claim_dict
return self._claim_dict
@property
def permanent_url(self) -> str:
if self.script.is_claim_involved:
return f"lbry://{self.claim_name}#{self.claim_id}"
if self.channel is not None:
return "{}#{}/{}".format(
self.channel.claim_name,
self.channel.claim_id,
self.claim_name
)
return f"{self.claim_name}#{self.claim_id}"
raise ValueError('No claim associated.')
@property
def has_private_key(self):
return self.private_key is not None
def is_signed_by(self, channel: 'Output', ledger=None):
if self.claim.unsigned_payload:
pieces = [
Base58.decode(self.get_address(ledger)),
self.claim.unsigned_payload,
self.claim.signing_channel_hash[::-1]
]
else:
pieces = [
self.tx_ref.tx.inputs[0].txo_ref.hash,
self.claim.signing_channel_hash,
self.claim.to_message_bytes()
]
digest = sha256(b''.join(pieces))
public_key = load_der_public_key(channel.claim.channel.public_key_bytes, default_backend())
hash = hashes.SHA256()
signature = hexlify(self.claim.signature)
r = int(signature[:int(len(signature)/2)], 16)
s = int(signature[int(len(signature)/2):], 16)
encoded_sig = sigencode_der(r, s, len(signature)*4)
public_key.verify(encoded_sig, digest, ec.ECDSA(Prehashed(hash)))
return True
def sign(self, channel: 'Output', first_input_id=None):
self.channel = channel
self.claim.signing_channel_hash = channel.claim_hash
digest = sha256(b''.join([
first_input_id or self.tx_ref.tx.inputs[0].txo_ref.hash,
self.claim.signing_channel_hash,
self.claim.to_message_bytes()
]))
private_key = ecdsa.SigningKey.from_pem(channel.private_key, hashfunc=hashlib.sha256)
self.claim.signature = private_key.sign_digest_deterministic(digest, hashfunc=hashlib.sha256)
self.script.generate()
def clear_signature(self):
self.channel = None
self.claim.clear_signature()
def generate_channel_private_key(self):
private_key = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1, hashfunc=hashlib.sha256)
self.private_key = private_key.to_pem().decode()
self.claim.channel.public_key_bytes = private_key.get_verifying_key().to_der()
self.script.generate()
return self.private_key
def is_channel_private_key(self, private_key_pem):
private_key = ecdsa.SigningKey.from_pem(private_key_pem, hashfunc=hashlib.sha256)
return self.claim.channel.public_key_bytes == private_key.get_verifying_key().to_der()
@classmethod
def pay_claim_name_pubkey_hash(
cls, amount: int, claim_name: str, claim: Claim, pubkey_hash: bytes) -> 'Output':
cls, amount: int, claim_name: str, claim: bytes, pubkey_hash: bytes) -> 'Output':
script = cls.script_class.pay_claim_name_pubkey_hash(
claim_name.encode(), claim, pubkey_hash)
txo = cls(amount, script)
return txo
@classmethod
def pay_update_claim_pubkey_hash(
cls, amount: int, claim_name: str, claim_id: str, claim: Claim, pubkey_hash: bytes) -> 'Output':
script = cls.script_class.pay_update_claim_pubkey_hash(
claim_name.encode(), unhexlify(claim_id)[::-1], claim, pubkey_hash)
txo = cls(amount, script)
return txo
@classmethod
def pay_support_pubkey_hash(cls, amount: int, claim_name: str, claim_id: str, pubkey_hash: bytes) -> 'Output':
script = cls.script_class.pay_support_pubkey_hash(claim_name.encode(), unhexlify(claim_id)[::-1], pubkey_hash)
return cls(amount, script)
@classmethod
@@ -163,15 +99,24 @@ class Output(BaseOutput):
script = cls.script_class.purchase_claim_pubkey_hash(unhexlify(claim_id)[::-1], pubkey_hash)
return cls(amount, script)
@classmethod
def pay_update_claim_pubkey_hash(
cls, amount: int, claim_name: str, claim_id: str, claim: bytes, pubkey_hash: bytes) -> 'Output':
script = cls.script_class.pay_update_claim_pubkey_hash(
claim_name.encode(), unhexlify(claim_id)[::-1], claim, pubkey_hash)
return cls(amount, script)
@classmethod
def pay_support_pubkey_hash(cls, amount: int, claim_name: str, claim_id: str, pubkey_hash: bytes) -> 'Output':
script = cls.script_class.pay_support_pubkey_hash(claim_name.encode(), unhexlify(claim_id)[::-1], pubkey_hash)
return cls(amount, script)
class Transaction(BaseTransaction):
input_class = Input
output_class = Output
outputs: ReadOnlyList[Output]
inputs: ReadOnlyList[Input]
@classmethod
def pay(cls, amount: int, address: bytes, funding_accounts: List[Account], change_account: Account):
ledger = cls.ensure_all_have_same_ledger(funding_accounts, change_account)
@@ -179,52 +124,46 @@ class Transaction(BaseTransaction):
return cls.create([], [output], funding_accounts, change_account)
@classmethod
def claim_create(
cls, name: str, claim: Claim, amount: int, holding_address: str,
funding_accounts: List[Account], change_account: Account, signing_channel: Output = None):
def claim(cls, name: str, meta: ClaimDict, amount: int, holding_address: bytes,
funding_accounts: List[Account], change_account: Account):
ledger = cls.ensure_all_have_same_ledger(funding_accounts, change_account)
claim_output = Output.pay_claim_name_pubkey_hash(
amount, name, claim, ledger.address_to_hash160(holding_address)
amount, name, meta.serialized, ledger.address_to_hash160(holding_address)
)
if signing_channel is not None:
claim_output.sign(signing_channel, b'placeholder txid:nout')
return cls.create([], [claim_output], funding_accounts, change_account, sign=False)
@classmethod
def claim_update(
cls, previous_claim: Output, claim: Claim, amount: int, holding_address: str,
funding_accounts: List[Account], change_account: Account, signing_channel: Output = None):
ledger = cls.ensure_all_have_same_ledger(funding_accounts, change_account)
updated_claim = Output.pay_update_claim_pubkey_hash(
amount, previous_claim.claim_name, previous_claim.claim_id,
claim, ledger.address_to_hash160(holding_address)
)
if signing_channel is not None:
updated_claim.sign(signing_channel, b'placeholder txid:nout')
else:
updated_claim.clear_signature()
return cls.create(
[Input.spend(previous_claim)], [updated_claim], funding_accounts, change_account, sign=False
)
@classmethod
def support(cls, claim_name: str, claim_id: str, amount: int, holding_address: str,
funding_accounts: List[Account], change_account: Account):
ledger = cls.ensure_all_have_same_ledger(funding_accounts, change_account)
support_output = Output.pay_support_pubkey_hash(
amount, claim_name, claim_id, ledger.address_to_hash160(holding_address)
)
return cls.create([], [support_output], funding_accounts, change_account)
return cls.create([], [claim_output], funding_accounts, change_account)
@classmethod
def purchase(cls, claim: Output, amount: int, merchant_address: bytes,
funding_accounts: List[Account], change_account: Account):
funding_accounts: List[Account], change_account: Account):
ledger = cls.ensure_all_have_same_ledger(funding_accounts, change_account)
claim_output = Output.purchase_claim_pubkey_hash(
amount, claim.claim_id, ledger.address_to_hash160(merchant_address)
)
return cls.create([], [claim_output], funding_accounts, change_account)
@classmethod
def update(cls, previous_claim: Output, meta: ClaimDict, amount: int, holding_address: bytes,
funding_accounts: List[Account], change_account: Account):
ledger = cls.ensure_all_have_same_ledger(funding_accounts, change_account)
updated_claim = Output.pay_update_claim_pubkey_hash(
amount, previous_claim.claim_name, previous_claim.claim_id,
meta.serialized, ledger.address_to_hash160(holding_address)
)
return cls.create([Input.spend(previous_claim)], [updated_claim], funding_accounts, change_account)
@classmethod
def support(cls, claim_name: str, claim_id: str, amount: int, holding_address: bytes,
funding_accounts: List[Account], change_account: Account):
ledger = cls.ensure_all_have_same_ledger(funding_accounts, change_account)
output = Output.pay_support_pubkey_hash(
amount, claim_name, claim_id, ledger.address_to_hash160(holding_address)
)
return cls.create([], [output], funding_accounts, change_account)
@classmethod
def abandon(cls, claims: Iterable[Output], funding_accounts: Iterable[Account], change_account: Account):
return cls.create([Input.spend(txo) for txo in claims], [], funding_accounts, change_account)
@property
def my_inputs(self):
for txi in self.inputs:
+5 -4
View File
@@ -1,5 +1,6 @@
build:
rm types/v2/* -rf
touch types/v2/__init__.py
cd types/v2/ && protoc --python_out=. -I ../../../../../types/v2/proto/ ../../../../../types/v2/proto/*.proto
sed -e 's/^import\ \(.*\)_pb2\ /from . import\ \1_pb2\ /g' -i types/v2/*.py
rm proto/* -rf
cd proto && protoc --python_out=. -I ../../../../types/proto/ ../../../../types/proto/*.proto
sed -e 's/^import\ \(.*\)_pb2\ /from . import\ \1_pb2\ /g' -i proto/*.py
touch proto/__init__.py
+13 -1
View File
@@ -1 +1,13 @@
from .claim import Claim
import os
__version__ = "0.0.16"
BLOCKCHAIN_NAME_ENVVAR = "LBRYSCHEMA_BLOCKCHAIN_NAME"
if BLOCKCHAIN_NAME_ENVVAR in os.environ:
if os.environ[BLOCKCHAIN_NAME_ENVVAR] in ['lbrycrd_main', 'lbrycrd_regtest',
'lbrycrd_testnet']:
BLOCKCHAIN_NAME = os.environ[BLOCKCHAIN_NAME_ENVVAR]
else:
raise OSError("invalid blockchain name: %s" % os.environ[BLOCKCHAIN_NAME_ENVVAR])
else:
BLOCKCHAIN_NAME = "lbrycrd_main"
+64
View File
@@ -0,0 +1,64 @@
import lbrynet.schema
from lbrynet.schema.base import b58encode, b58decode, validate_b58_checksum
from lbrynet.schema.hashing import double_sha256, hash160
from lbrynet.schema.error import InvalidAddress
from lbrynet.schema.schema import ADDRESS_LENGTH, ADDRESS_PREFIXES, PUBKEY_ADDRESS, SCRIPT_ADDRESS
def validate_address_length(addr_bytes):
if len(addr_bytes) != ADDRESS_LENGTH:
raise InvalidAddress("Invalid address length: %i" % len(addr_bytes))
def validate_address_prefix(addr_bytes):
prefix = addr_bytes[0]
if prefix not in ADDRESS_PREFIXES[lbrynet.schema.BLOCKCHAIN_NAME].values():
raise InvalidAddress("Invalid address prefix: %.2X" % prefix)
def validate_lbrycrd_address_bytes(addr_bytes):
validate_address_length(addr_bytes)
validate_address_prefix(addr_bytes)
validate_b58_checksum(addr_bytes)
return addr_bytes
def decode_address(v):
"""decode and validate a b58 address"""
return validate_lbrycrd_address_bytes(b58decode(v))
def encode_address(addr_bytes):
"""validate and encode an address as b58"""
v = validate_lbrycrd_address_bytes(addr_bytes)
return b58encode(v)
def hash_160_bytes_to_address(h160, addrtype=PUBKEY_ADDRESS):
if addrtype == PUBKEY_ADDRESS:
prefix = chr(ADDRESS_PREFIXES[lbrynet.schema.BLOCKCHAIN_NAME][PUBKEY_ADDRESS])
elif addrtype == SCRIPT_ADDRESS:
prefix = chr(ADDRESS_PREFIXES[lbrynet.schema.BLOCKCHAIN_NAME][SCRIPT_ADDRESS])
else:
raise Exception("Invalid address prefix")
return b58encode(prefix + h160 + double_sha256(prefix + h160)[0:4])
def public_key_to_address(public_key):
return hash_160_bytes_to_address(hash160(public_key))
def address_to_hash_160(addr):
bytes = decode_address(addr)
prefix, pubkey_bytes, addr_checksum = bytes[0], bytes[1:21], bytes[21:]
if prefix == chr(ADDRESS_PREFIXES[lbrynet.schema.BLOCKCHAIN_NAME][PUBKEY_ADDRESS]):
return PUBKEY_ADDRESS, pubkey_bytes
return SCRIPT_ADDRESS, pubkey_bytes
def is_address(addr):
try:
addr_bytes = decode_address(addr)
return True
except InvalidAddress:
return False
-521
View File
@@ -1,521 +0,0 @@
import json
import logging
import os.path
import hashlib
from typing import Tuple, List
from string import ascii_letters
from decimal import Decimal, ROUND_UP
from binascii import hexlify, unhexlify
from google.protobuf.json_format import MessageToDict
from torba.client.hash import Base58
from torba.client.constants import COIN
from lbrynet.schema.mime_types import guess_media_type
from lbrynet.schema.base import Metadata, BaseMessageList
from lbrynet.schema.types.v2.claim_pb2 import (
Fee as FeeMessage,
Location as LocationMessage,
Language as LanguageMessage
)
log = logging.getLogger(__name__)
def calculate_sha384_file_hash(file_path):
sha384 = hashlib.sha384()
with open(file_path, 'rb') as f:
for chunk in iter(lambda: f.read(128 * sha384.block_size), b''):
sha384.update(chunk)
return sha384.digest()
class Dimmensional(Metadata):
__slots__ = ()
@property
def width(self) -> int:
return self.message.width
@width.setter
def width(self, width: int):
self.message.width = width
@property
def height(self) -> int:
return self.message.height
@height.setter
def height(self, height: int):
self.message.height = height
@property
def dimensions(self) -> Tuple[int, int]:
return self.width, self.height
@dimensions.setter
def dimensions(self, dimensions: Tuple[int, int]):
self.message.width, self.message.height = dimensions
def _extract(self, file_metadata, field):
try:
setattr(self, field, file_metadata.getValues(field)[0])
except:
log.exception(f'Could not extract {field} from file metadata.')
def update(self, file_metadata=None, height=None, width=None):
if height is not None:
self.height = height
elif file_metadata:
self._extract(file_metadata, 'height')
if width is not None:
self.width = width
elif file_metadata:
self._extract(file_metadata, 'width')
class Playable(Metadata):
__slots__ = ()
@property
def duration(self) -> int:
return self.message.duration
@duration.setter
def duration(self, duration: int):
self.message.duration = duration
def update(self, file_metadata=None, duration=None):
if duration is not None:
self.duration = duration
elif file_metadata:
try:
self.duration = file_metadata.getValues('duration')[0].seconds
except:
log.exception('Could not extract duration from file metadata.')
class Image(Dimmensional):
__slots__ = ()
class Audio(Playable):
__slots__ = ()
class Video(Dimmensional, Playable):
__slots__ = ()
def update(self, file_metadata=None, height=None, width=None, duration=None):
Dimmensional.update(self, file_metadata, height, width)
Playable.update(self, file_metadata, duration)
class Source(Metadata):
__slots__ = ()
def update(self, file_path=None):
if file_path is not None:
self.name = os.path.basename(file_path)
self.media_type, stream_type = guess_media_type(file_path)
if not os.path.isfile(file_path):
raise Exception(f"File does not exist: {file_path}")
self.size = os.path.getsize(file_path)
if self.size == 0:
raise Exception(f"Cannot publish empty file: {file_path}")
self.file_hash_bytes = calculate_sha384_file_hash(file_path)
return stream_type
@property
def name(self) -> str:
return self.message.name
@name.setter
def name(self, name: str):
self.message.name = name
@property
def size(self) -> int:
return self.message.size
@size.setter
def size(self, size: int):
self.message.size = size
@property
def media_type(self) -> str:
return self.message.media_type
@media_type.setter
def media_type(self, media_type: str):
self.message.media_type = media_type
@property
def file_hash(self) -> str:
return hexlify(self.message.hash).decode()
@file_hash.setter
def file_hash(self, file_hash: str):
self.message.hash = unhexlify(file_hash.encode())
@property
def file_hash_bytes(self) -> bytes:
return self.message.hash
@file_hash_bytes.setter
def file_hash_bytes(self, file_hash_bytes: bytes):
self.message.hash = file_hash_bytes
@property
def sd_hash(self) -> str:
return hexlify(self.message.sd_hash).decode()
@sd_hash.setter
def sd_hash(self, sd_hash: str):
self.message.sd_hash = unhexlify(sd_hash.encode())
@property
def sd_hash_bytes(self) -> bytes:
return self.message.sd_hash
@sd_hash_bytes.setter
def sd_hash_bytes(self, sd_hash: bytes):
self.message.sd_hash = sd_hash
@property
def url(self) -> str:
return self.message.url
@url.setter
def url(self, url: str):
self.message.url = url
class Fee(Metadata):
__slots__ = ()
def update(self, address: str = None, currency: str = None, amount=None):
if address is not None:
self.address = address
if currency is not None and amount is not None:
currency = currency.lower()
assert currency in ('lbc', 'btc', 'usd'), f'Unknown currency type: {currency}'
setattr(self, currency, Decimal(amount))
@property
def currency(self) -> str:
return FeeMessage.Currency.Name(self.message.currency)
@property
def address(self) -> str:
return Base58.encode(self.message.address)
@address.setter
def address(self, address: str):
self.message.address = Base58.decode(address)
@property
def address_bytes(self) -> bytes:
return self.message.address
@address_bytes.setter
def address_bytes(self, address: bytes):
self.message.address = address
@property
def amount(self) -> Decimal:
if self.currency == 'LBC':
return self.lbc
if self.currency == 'BTC':
return self.btc
if self.currency == 'USD':
return self.usd
DEWIES = Decimal(COIN)
@property
def lbc(self) -> Decimal:
if self.message.currency != FeeMessage.LBC:
raise ValueError('LBC can only be returned for LBC fees.')
return Decimal(self.message.amount / self.DEWIES)
@lbc.setter
def lbc(self, amount: Decimal):
self.dewies = int(amount * self.DEWIES)
@property
def dewies(self) -> int:
if self.message.currency != FeeMessage.LBC:
raise ValueError('Dewies can only be returned for LBC fees.')
return self.message.amount
@dewies.setter
def dewies(self, amount: int):
self.message.amount = amount
self.message.currency = FeeMessage.LBC
SATOSHIES = Decimal(COIN)
@property
def btc(self) -> Decimal:
if self.message.currency != FeeMessage.BTC:
raise ValueError('BTC can only be returned for BTC fees.')
return Decimal(self.message.amount / self.SATOSHIES)
@btc.setter
def btc(self, amount: Decimal):
self.satoshis = int(amount * self.SATOSHIES)
@property
def satoshis(self) -> int:
if self.message.currency != FeeMessage.BTC:
raise ValueError('Satoshies can only be returned for BTC fees.')
return self.message.amount
@satoshis.setter
def satoshis(self, amount: int):
self.message.amount = amount
self.message.currency = FeeMessage.BTC
PENNIES = Decimal('100.0')
PENNY = Decimal('0.01')
@property
def usd(self) -> Decimal:
if self.message.currency != FeeMessage.USD:
raise ValueError('USD can only be returned for USD fees.')
return Decimal(self.message.amount / self.PENNIES)
@usd.setter
def usd(self, amount: Decimal):
self.pennies = int(amount.quantize(self.PENNY, ROUND_UP) * self.PENNIES)
@property
def pennies(self) -> int:
if self.message.currency != FeeMessage.USD:
raise ValueError('Pennies can only be returned for USD fees.')
return self.message.amount
@pennies.setter
def pennies(self, amount: int):
self.message.amount = amount
self.message.currency = FeeMessage.USD
class ClaimReference(Metadata):
__slots__ = ()
@property
def claim_id(self) -> str:
return hexlify(self.claim_hash[::-1]).decode()
@claim_id.setter
def claim_id(self, claim_id: str):
self.claim_hash = unhexlify(claim_id)[::-1]
@property
def claim_hash(self) -> bytes:
return self.message.claim_hash
@claim_hash.setter
def claim_hash(self, claim_hash: bytes):
self.message.claim_hash = claim_hash
class ClaimList(BaseMessageList[ClaimReference]):
__slots__ = ()
item_class = ClaimReference
@property
def _message(self):
return self.message.claim_references
def append(self, value):
self.add().claim_id = value
@property
def ids(self) -> List[str]:
return [c.claim_id for c in self]
class Language(Metadata):
__slots__ = ()
@property
def langtag(self) -> str:
langtag = []
if self.language:
langtag.append(self.language)
if self.script:
langtag.append(self.script)
if self.region:
langtag.append(self.region)
return '-'.join(langtag)
@langtag.setter
def langtag(self, langtag: str):
parts = langtag.split('-')
self.language = parts.pop(0)
if parts and len(parts[0]) == 4:
self.script = parts.pop(0)
if parts and len(parts[0]) == 2:
self.region = parts.pop(0)
assert not parts, f"Failed to parse language tag: {langtag}"
@property
def language(self) -> str:
if self.message.language:
return LanguageMessage.Language.Name(self.message.language)
@language.setter
def language(self, language: str):
self.message.language = LanguageMessage.Language.Value(language)
@property
def script(self) -> str:
if self.message.script:
return LanguageMessage.Script.Name(self.message.script)
@script.setter
def script(self, script: str):
self.message.script = LanguageMessage.Script.Value(script)
@property
def region(self) -> str:
if self.message.region:
return LocationMessage.Country.Name(self.message.region)
@region.setter
def region(self, region: str):
self.message.region = LocationMessage.Country.Value(region)
class LanguageList(BaseMessageList[Language]):
__slots__ = ()
item_class = Language
def append(self, value: str):
self.add().langtag = value
class Location(Metadata):
__slots__ = ()
def from_value(self, value):
if isinstance(value, str) and value.startswith('{'):
value = json.loads(value)
if isinstance(value, dict):
for key, val in value.items():
setattr(self, key, val)
elif isinstance(value, str):
parts = value.split(':')
if len(parts) > 2 or (parts[0] and parts[0][0] in ascii_letters):
country = parts and parts.pop(0)
if country:
self.country = country
state = parts and parts.pop(0)
if state:
self.state = state
city = parts and parts.pop(0)
if city:
self.city = city
code = parts and parts.pop(0)
if code:
self.code = code
latitude = parts and parts.pop(0)
if latitude:
self.latitude = latitude
longitude = parts and parts.pop(0)
if longitude:
self.longitude = longitude
else:
raise ValueError(f'Could not parse country value: {value}')
def to_dict(self):
d = MessageToDict(self.message)
if self.message.longitude:
d['longitude'] = self.longitude
if self.message.latitude:
d['latitude'] = self.latitude
return d
@property
def country(self) -> str:
if self.message.country:
return LocationMessage.Country.Name(self.message.country)
@country.setter
def country(self, country: str):
self.message.country = LocationMessage.Country.Value(country)
@property
def state(self) -> str:
return self.message.state
@state.setter
def state(self, state: str):
self.message.state = state
@property
def city(self) -> str:
return self.message.city
@city.setter
def city(self, city: str):
self.message.city = city
@property
def code(self) -> str:
return self.message.code
@code.setter
def code(self, code: str):
self.message.code = code
GPS_PRECISION = Decimal('10000000')
@property
def latitude(self) -> str:
if self.message.latitude:
return str(Decimal(self.message.latitude) / self.GPS_PRECISION)
@latitude.setter
def latitude(self, latitude: str):
latitude = Decimal(latitude)
assert -90 <= latitude <= 90, "Latitude must be between -90 and 90 degrees."
self.message.latitude = int(latitude * self.GPS_PRECISION)
@property
def longitude(self) -> str:
if self.message.longitude:
return str(Decimal(self.message.longitude) / self.GPS_PRECISION)
@longitude.setter
def longitude(self, longitude: str):
longitude = Decimal(longitude)
assert -180 <= longitude <= 180, "Longitude must be between -180 and 180 degrees."
self.message.longitude = int(longitude * self.GPS_PRECISION)
class LocationList(BaseMessageList[Location]):
__slots__ = ()
item_class = Location
def append(self, value):
self.add().from_value(value)
+72 -103
View File
@@ -1,121 +1,90 @@
from binascii import hexlify, unhexlify
from typing import List, Iterator, TypeVar, Generic
from google.protobuf.message import DecodeError
from google.protobuf.json_format import MessageToDict
from lbrynet.schema.schema import ADDRESS_CHECKSUM_LENGTH
from lbrynet.schema.hashing import double_sha256
from lbrynet.schema.error import InvalidAddress
class Signable:
__slots__ = (
'message', 'version', 'signature',
'signature_type', 'unsigned_payload', 'signing_channel_hash'
)
message_class = None
def __init__(self, message=None):
self.message = message or self.message_class()
self.version = 2
self.signature = None
self.signature_type = 'SECP256k1'
self.unsigned_payload = None
self.signing_channel_hash = None
def clear_signature(self):
self.signature = None
self.unsigned_payload = None
self.signing_channel_hash = None
@property
def signing_channel_id(self):
return hexlify(self.signing_channel_hash[::-1]).decode() if self.signing_channel_hash else None
@signing_channel_id.setter
def signing_channel_id(self, channel_id: str):
self.signing_channel_hash = unhexlify(channel_id)[::-1]
@property
def is_signed(self):
return self.signature is not None
def to_dict(self):
return MessageToDict(self.message)
def to_message_bytes(self) -> bytes:
return self.message.SerializeToString()
def to_bytes(self) -> bytes:
pieces = bytearray()
if self.is_signed:
pieces.append(1)
pieces.extend(self.signing_channel_hash)
pieces.extend(self.signature)
else:
pieces.append(0)
pieces.extend(self.to_message_bytes())
return bytes(pieces)
@classmethod
def from_bytes(cls, data: bytes):
signable = cls()
if data[0] == 0:
signable.message.ParseFromString(data[1:])
elif data[0] == 1:
signable.signing_channel_hash = data[1:21]
signable.signature = data[21:85]
signable.message.ParseFromString(data[85:])
else:
raise DecodeError('Could not determine message format version.')
return signable
def __len__(self):
return len(self.to_bytes())
def __bytes__(self):
return self.to_bytes()
alphabet = b'123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'
class Metadata:
__slots__ = 'message',
def __init__(self, message):
self.message = message
def scrub_input(v):
if isinstance(v, str) and not isinstance(v, bytes):
v = v.encode('ascii')
return v
I = TypeVar('I')
def b58encode_int(i, default_one=True):
'''Encode an integer using Base58'''
if not i and default_one:
return alphabet[0:1]
string = b""
while i:
i, idx = divmod(i, 58)
string = alphabet[idx:idx+1] + string
return string
class BaseMessageList(Metadata, Generic[I]):
def b58encode(v):
'''Encode a string using Base58'''
__slots__ = ()
v = scrub_input(v)
item_class = None
nPad = len(v)
v = v.lstrip(b'\0')
nPad -= len(v)
@property
def _message(self):
return self.message
p, acc = 1, 0
for c in reversed(v):
acc += p * c
p = p << 8
def add(self) -> I:
return self.item_class(self._message.add())
result = b58encode_int(acc, default_one=False)
def extend(self, values: List[str]):
for value in values:
self.append(value)
return alphabet[0:1] * nPad + result
def append(self, value: str):
raise NotImplemented
def __len__(self):
return len(self._message)
def b58decode_int(v):
'''Decode a Base58 encoded string as an integer'''
def __iter__(self) -> Iterator[I]:
for item in self._message:
yield self.item_class(item)
v = scrub_input(v)
def __getitem__(self, item) -> I:
return self.item_class(self._message[item])
decimal = 0
for char in v:
decimal = decimal * 58 + alphabet.index(char)
return decimal
def __delitem__(self, key):
del self._message[key]
def b58decode(v):
'''Decode a Base58 encoded string'''
v = scrub_input(v)
origlen = len(v)
v = v.lstrip(alphabet[0:1])
newlen = len(v)
acc = b58decode_int(v)
result = []
while acc > 0:
acc, mod = divmod(acc, 256)
result.append(mod)
return b'\0' * (origlen - newlen) + bytes(reversed(result))
def validate_b58_checksum(addr_bytes):
addr_without_checksum = addr_bytes[:-ADDRESS_CHECKSUM_LENGTH]
addr_checksum = addr_bytes[-ADDRESS_CHECKSUM_LENGTH:]
if double_sha256(addr_without_checksum)[:ADDRESS_CHECKSUM_LENGTH] != addr_checksum:
raise InvalidAddress("Invalid address checksum")
def b58decode_strip_checksum(v):
addr_bytes = b58decode(v)
validate_b58_checksum(addr_bytes)
return addr_bytes[:-ADDRESS_CHECKSUM_LENGTH]
def b58encode_with_checksum(addr_bytes):
addr_checksum = double_sha256(addr_bytes)[:ADDRESS_CHECKSUM_LENGTH]
return b58encode(addr_bytes + addr_checksum)
+171 -371
View File
@@ -1,398 +1,198 @@
import logging
from typing import List
from binascii import hexlify, unhexlify
import json
import binascii
from google.protobuf import json_format # pylint: disable=no-name-in-module
from google.protobuf.message import DecodeError as DecodeError_pb # pylint: disable=no-name-in-module,import-error
from google.protobuf.json_format import MessageToDict
from google.protobuf.message import DecodeError
from hachoir.core.log import log as hachoir_log
from hachoir.parser import createParser as binary_file_parser
from hachoir.metadata import extractMetadata as binary_file_metadata
from collections import OrderedDict
from lbrynet.schema import compat
from lbrynet.schema.base import Signable
from lbrynet.schema.mime_types import guess_media_type, guess_stream_type
from lbrynet.schema.attrs import (
Source, Playable, Dimmensional, Fee, Image, Video, Audio,
LanguageList, LocationList, ClaimList, ClaimReference
)
from lbrynet.schema.types.v2.claim_pb2 import Claim as ClaimMessage
from lbrynet.schema.schema.claim import Claim
from lbrynet.schema.proto import claim_pb2
from lbrynet.schema.signature import Signature
from lbrynet.schema.validator import get_validator
from lbrynet.schema.signer import get_signer
from lbrynet.schema.schema import NIST256p, CURVE_NAMES, CLAIM_TYPE_NAMES, SECP256k1
from lbrynet.schema.encoding import decode_fields, decode_b64_fields, encode_fields
from lbrynet.schema.error import DecodeError
from lbrynet.schema.fee import Fee
hachoir_log.use_print = False
log = logging.getLogger(__name__)
class Claim(Signable):
STREAM = 'stream'
CHANNEL = 'channel'
COLLECTION = 'collection'
REPOST = 'repost'
__slots__ = 'version',
message_class = ClaimMessage
def __init__(self, message=None):
super().__init__(message)
self.version = 2
class ClaimDict(OrderedDict):
def __init__(self, claim_dict=None, detached_signature: Signature=None):
if isinstance(claim_dict, claim_pb2.Claim):
raise Exception("To initialize %s with a Claim protobuf use %s.load_protobuf" %
(self.__class__.__name__, self.__class__.__name__))
self.detached_signature = detached_signature
OrderedDict.__init__(self, claim_dict or [])
@property
def claim_type(self) -> str:
return self.message.WhichOneof('type')
def protobuf_dict(self):
"""Claim dictionary using base64 to represent bytes"""
def get_message(self, type_name):
message = getattr(self.message, type_name)
if self.claim_type is None:
message.SetInParent()
if self.claim_type != type_name:
raise ValueError(f'Claim is not a {type_name}.')
return message
return json.loads(json_format.MessageToJson(self.protobuf, True))
@property
def protobuf(self):
"""Claim message object"""
return Claim.load(self)
@property
def serialized(self):
"""Serialized Claim protobuf"""
if self.detached_signature:
return self.detached_signature.serialized
return self.protobuf.SerializeToString()
@property
def serialized_no_signature(self):
"""Serialized Claim protobuf without publisherSignature field"""
claim = self.protobuf
claim.ClearField("publisherSignature")
return ClaimDict.load_protobuf(claim).serialized
@property
def has_signature(self):
return self.protobuf.HasField("publisherSignature") or (
self.detached_signature and self.detached_signature.raw_signature
)
@property
def is_certificate(self):
claim = self.protobuf
return CLAIM_TYPE_NAMES[claim.claimType] == "certificate"
@property
def is_stream(self):
return self.claim_type == self.STREAM
claim = self.protobuf
return CLAIM_TYPE_NAMES[claim.claimType] == "stream"
@property
def stream(self) -> 'Stream':
return Stream(self)
def source_hash(self):
claim = self.protobuf
if not CLAIM_TYPE_NAMES[claim.claimType] == "stream":
return None
return binascii.hexlify(claim.stream.source.source)
@property
def is_channel(self):
return self.claim_type == self.CHANNEL
def has_fee(self):
claim = self.protobuf
if not CLAIM_TYPE_NAMES[claim.claimType] == "stream":
return None
if claim.stream.metadata.HasField("fee"):
return True
return False
@property
def channel(self) -> 'Channel':
return Channel(self)
def source_fee(self):
claim = self.protobuf
if not CLAIM_TYPE_NAMES[claim.claimType] == "stream":
return None
if claim.stream.metadata.HasField("fee"):
return Fee.load_protobuf(claim.stream.metadata.fee)
return None
@property
def is_repost(self):
return self.claim_type == self.REPOST
def certificate_id(self) -> str:
if self.protobuf.HasField("publisherSignature"):
return binascii.hexlify(self.protobuf.publisherSignature.certificateId).decode()
if self.detached_signature and self.detached_signature.certificate_id:
return binascii.hexlify(self.detached_signature.certificate_id).decode()
@property
def repost(self) -> 'Repost':
return Repost(self)
def signature(self):
if not self.has_signature:
return None
return binascii.hexlify(self.protobuf.publisherSignature.signature)
@property
def is_collection(self):
return self.claim_type == self.COLLECTION
def protobuf_len(self):
"""Length of serialized string"""
return self.protobuf.ByteSize()
@property
def collection(self) -> 'Collection':
return Collection(self)
def json_len(self):
"""Length of json encoded string"""
return len(json.dumps(self.claim_dict))
@property
def claim_dict(self):
"""Claim dictionary with bytes represented as hex and base58"""
return dict(encode_fields(self, self.detached_signature))
@classmethod
def from_bytes(cls, data: bytes) -> 'Claim':
def load_protobuf_dict(cls, protobuf_dict, detached_signature=None):
"""
Load a ClaimDict from a dictionary with base64 encoded bytes
(as returned by the protobuf json formatter)
"""
return cls(decode_b64_fields(protobuf_dict), detached_signature=detached_signature)
@classmethod
def load_protobuf(cls, protobuf_claim, detached_signature=None):
"""Load ClaimDict from a protobuf Claim message"""
return cls.load_protobuf_dict(json.loads(json_format.MessageToJson(protobuf_claim, True)), detached_signature)
@classmethod
def load_dict(cls, claim_dict):
"""Load ClaimDict from a dictionary with hex and base58 encoded bytes"""
try:
return super().from_bytes(data)
except DecodeError:
claim = cls()
if data[0] == ord('{'):
claim.version = 0
compat.from_old_json_schema(claim, data)
elif data[0] not in (0, 1):
claim.version = 1
compat.from_types_v1(claim, data)
else:
raise
return claim
class BaseClaim:
__slots__ = 'claim', 'message'
claim_type = None
object_fields = 'thumbnail',
repeat_fields = 'tags', 'languages', 'locations'
def __init__(self, claim: Claim = None):
self.claim = claim or Claim()
self.message = self.claim.get_message(self.claim_type)
def to_dict(self):
claim = MessageToDict(self.claim.message, preserving_proto_field_name=True)
claim.update(claim.pop(self.claim_type))
if 'languages' in claim:
claim['languages'] = self.langtags
if 'locations' in claim:
claim['locations'] = [l.to_dict() for l in self.locations]
return claim
def update(self, **kwargs):
for key in list(kwargs):
for field in self.object_fields:
if key.startswith(f'{field}_'):
attr = getattr(self, field)
setattr(attr, key[len(f'{field}_'):], kwargs.pop(key))
continue
for l in self.repeat_fields:
field = getattr(self, l)
if kwargs.pop(f'clear_{l}', False):
del field[:]
items = kwargs.pop(l, None)
if items is not None:
if isinstance(items, str):
field.append(items)
elif isinstance(items, list):
field.extend(items)
else:
raise ValueError(f"Unknown {l} value: {items}")
for key, value in kwargs.items():
setattr(self, key, value)
@property
def title(self) -> str:
return self.claim.message.title
@title.setter
def title(self, title: str):
self.claim.message.title = title
@property
def description(self) -> str:
return self.claim.message.description
@description.setter
def description(self, description: str):
self.claim.message.description = description
@property
def thumbnail(self) -> Source:
return Source(self.claim.message.thumbnail)
@property
def tags(self) -> List:
return self.claim.message.tags
@property
def languages(self) -> LanguageList:
return LanguageList(self.claim.message.languages)
@property
def langtags(self) -> List[str]:
return [l.langtag for l in self.languages]
@property
def locations(self) -> LocationList:
return LocationList(self.claim.message.locations)
class Stream(BaseClaim):
__slots__ = ()
claim_type = Claim.STREAM
object_fields = BaseClaim.object_fields + ('source',)
def to_dict(self):
claim = super().to_dict()
if 'source' in claim:
if 'hash' in claim['source']:
claim['source']['hash'] = self.source.file_hash
if 'sd_hash' in claim['source']:
claim['source']['sd_hash'] = self.source.sd_hash
fee = claim.get('fee', {})
if 'address' in fee:
fee['address'] = self.fee.address
if 'amount' in fee:
fee['amount'] = str(self.fee.amount)
stream_type = self.message.WhichOneof('type')
if stream_type:
claim['stream_type'] = stream_type
return claim
def update(self, file_path=None, height=None, width=None, duration=None, **kwargs):
if kwargs.pop('clear_fee', False):
self.message.ClearField('fee')
else:
self.fee.update(
kwargs.pop('fee_address', None),
kwargs.pop('fee_currency', None),
kwargs.pop('fee_amount', None)
)
if 'sd_hash' in kwargs:
self.source.sd_hash = kwargs.pop('sd_hash')
if 'file_name' in kwargs:
self.source.name = kwargs.pop('file_name')
if 'file_hash' in kwargs:
self.source.file_hash = kwargs.pop('file_hash')
stream_type = None
if file_path is not None:
stream_type = self.source.update(file_path=file_path)
elif self.source.name:
self.source.media_type, stream_type = guess_media_type(self.source.name)
elif self.source.media_type:
stream_type = guess_stream_type(self.source.media_type)
if 'file_size' in kwargs:
self.source.size = kwargs.pop('file_size')
if stream_type in ('image', 'video', 'audio'):
media = getattr(self, stream_type)
media_args = {'file_metadata': None}
if file_path is not None:
try:
media_args['file_metadata'] = binary_file_metadata(binary_file_parser(file_path))
except:
log.exception('Could not read file metadata.')
if isinstance(media, Playable):
media_args['duration'] = duration
if isinstance(media, Dimmensional):
media_args['height'] = height
media_args['width'] = width
media.update(**media_args)
super().update(**kwargs)
@property
def author(self) -> str:
return self.message.author
@author.setter
def author(self, author: str):
self.message.author = author
@property
def license(self) -> str:
return self.message.license
@license.setter
def license(self, license: str):
self.message.license = license
@property
def license_url(self) -> str:
return self.message.license_url
@license_url.setter
def license_url(self, license_url: str):
self.message.license_url = license_url
@property
def release_time(self) -> int:
return self.message.release_time
@release_time.setter
def release_time(self, release_time: int):
self.message.release_time = release_time
@property
def fee(self) -> Fee:
return Fee(self.message.fee)
@property
def has_fee(self) -> bool:
return self.message.HasField('fee')
@property
def source(self) -> Source:
return Source(self.message.source)
@property
def stream_type(self) -> str:
return self.message.WhichOneof('type')
@property
def image(self) -> Image:
return Image(self.message.image)
@property
def video(self) -> Video:
return Video(self.message.video)
@property
def audio(self) -> Audio:
return Audio(self.message.audio)
class Channel(BaseClaim):
__slots__ = ()
claim_type = Claim.CHANNEL
object_fields = BaseClaim.object_fields + ('cover',)
repeat_fields = BaseClaim.repeat_fields + ('featured',)
def to_dict(self):
claim = super().to_dict()
claim['public_key'] = self.public_key
if 'featured' in claim:
claim['featured'] = self.featured.ids
return claim
@property
def public_key(self) -> str:
return hexlify(self.message.public_key).decode()
@public_key.setter
def public_key(self, sd_public_key: str):
self.message.public_key = unhexlify(sd_public_key.encode())
@property
def public_key_bytes(self) -> bytes:
return self.message.public_key
@public_key_bytes.setter
def public_key_bytes(self, public_key: bytes):
self.message.public_key = public_key
@property
def email(self) -> str:
return self.message.email
@email.setter
def email(self, email: str):
self.message.email = email
@property
def website_url(self) -> str:
return self.message.website_url
@website_url.setter
def website_url(self, website_url: str):
self.message.website_url = website_url
@property
def cover(self) -> Source:
return Source(self.message.cover)
@property
def featured(self) -> ClaimList:
return ClaimList(self.message.featured)
class Repost(BaseClaim):
__slots__ = ()
claim_type = Claim.REPOST
@property
def reference(self) -> ClaimReference:
return ClaimReference(self.message)
class Collection(BaseClaim):
__slots__ = ()
claim_type = Claim.COLLECTION
repeat_fields = BaseClaim.repeat_fields + ('claims',)
def to_dict(self):
claim = super().to_dict()
if 'claim_references' in claim:
claim['claim_references'] = self.claims.ids
return claim
@property
def claims(self) -> ClaimList:
return ClaimList(self.message)
claim_dict, detached_signature = decode_fields(claim_dict)
return cls.load_protobuf(cls(claim_dict).protobuf, detached_signature)
except json_format.ParseError as err:
raise DecodeError(str(err))
@classmethod
def deserialize(cls, serialized):
"""Load a ClaimDict from a serialized protobuf string"""
detached_signature = Signature.flagged_parse(serialized)
temp_claim = claim_pb2.Claim()
try:
temp_claim.ParseFromString(detached_signature.payload)
except DecodeError_pb:
raise DecodeError(DecodeError_pb)
return cls.load_protobuf(temp_claim, detached_signature=detached_signature)
@classmethod
def generate_certificate(cls, private_key, curve=SECP256k1):
signer = get_signer(curve).load_pem(private_key)
return cls.load_protobuf(signer.certificate)
def sign(self, private_key, claim_address, cert_claim_id, curve=SECP256k1, name=None, force_detached=False):
signer = get_signer(curve).load_pem(private_key)
signed, signature = signer.sign_stream_claim(self, claim_address, cert_claim_id, name, force_detached)
return ClaimDict.load_protobuf(signed, signature)
def validate_signature(self, claim_address, certificate, name=None):
if isinstance(certificate, ClaimDict):
certificate = certificate.protobuf
curve = CURVE_NAMES[certificate.certificate.keyType]
validator = get_validator(curve).load_from_certificate(certificate, self.certificate_id)
return validator.validate_claim_signature(self, claim_address, name)
def validate_private_key(self, private_key, certificate_id):
certificate = self.protobuf
if CLAIM_TYPE_NAMES[certificate.claimType] != "certificate":
return
curve = CURVE_NAMES[certificate.certificate.keyType]
validator = get_validator(curve).load_from_certificate(certificate, certificate_id)
signing_key = validator.signing_key_from_pem(private_key)
return validator.validate_private_key(signing_key)
def get_validator(self, certificate_id):
"""
Get a lbrynet.schema.validator.Validator object for a certificate claim
:param certificate_id: claim id of this certificate claim
:return: None or lbrynet.schema.validator.Validator object
"""
claim = self.protobuf
if CLAIM_TYPE_NAMES[claim.claimType] != "certificate":
return
curve = CURVE_NAMES[claim.certificate.keyType]
return get_validator(curve).load_from_certificate(claim, certificate_id)
-91
View File
@@ -1,91 +0,0 @@
import json
from decimal import Decimal
from google.protobuf.message import DecodeError
from lbrynet.schema.types.v1.legacy_claim_pb2 import Claim as OldClaimMessage
from lbrynet.schema.types.v1.certificate_pb2 import KeyType
from lbrynet.schema.types.v1.fee_pb2 import Fee as FeeMessage
def from_old_json_schema(claim, payload: bytes):
try:
value = json.loads(payload)
except:
raise DecodeError('Could not parse JSON.')
stream = claim.stream
stream.source.sd_hash = value['sources']['lbry_sd_hash']
stream.source.media_type = value.get(
'content_type', value.get('content-type', 'application/octet-stream'))
stream.title = value.get('title', '')
stream.description = value.get('description', '')
if value.get('thumbnail', ''):
stream.thumbnail.url = value.get('thumbnail', '')
stream.author = value.get('author', '')
stream.license = value.get('license', '')
stream.license_url = value.get('license_url', '')
language = value.get('language', '')
if language:
if language.lower() == 'english':
language = 'en'
try:
stream.languages.append(language)
except:
pass
if value.get('nsfw', False):
stream.tags.append('mature')
if "fee" in value:
fee = value["fee"]
currency = list(fee.keys())[0]
if currency == 'LBC':
stream.fee.lbc = Decimal(fee[currency]['amount'])
elif currency == 'USD':
stream.fee.usd = Decimal(fee[currency]['amount'])
elif currency == 'BTC':
stream.fee.btc = Decimal(fee[currency]['amount'])
else:
raise DecodeError(f'Unknown currency: {currency}')
stream.fee.address = fee[currency]['address']
return claim
def from_types_v1(claim, payload: bytes):
old = OldClaimMessage()
old.ParseFromString(payload)
if old.claimType == 2:
channel = claim.channel
channel.public_key_bytes = old.certificate.publicKey
else:
stream = claim.stream
stream.title = old.stream.metadata.title
stream.description = old.stream.metadata.description
stream.author = old.stream.metadata.author
stream.license = old.stream.metadata.license
stream.license_url = old.stream.metadata.licenseUrl
stream.thumbnail.url = old.stream.metadata.thumbnail
if old.stream.metadata.HasField('language'):
stream.languages.add().message.language = old.stream.metadata.language
stream.source.media_type = old.stream.source.contentType
stream.source.sd_hash_bytes = old.stream.source.source
if old.stream.metadata.nsfw:
stream.tags.append('mature')
if old.stream.metadata.HasField('fee'):
fee = old.stream.metadata.fee
stream.fee.address_bytes = fee.address
currency = FeeMessage.Currency.Name(fee.currency)
if currency == 'LBC':
stream.fee.lbc = Decimal(fee.amount)
elif currency == 'USD':
stream.fee.usd = Decimal(fee.amount)
elif currency == 'BTC':
stream.fee.btc = Decimal(fee.amount)
else:
raise DecodeError(f'Unsupported currency: {currency}')
if old.HasField('publisherSignature'):
sig = old.publisherSignature
claim.signature = sig.signature
claim.signature_type = KeyType.Name(sig.signatureType)
claim.signing_channel_hash = sig.certificateId[::-1]
old.ClearField("publisherSignature")
claim.unsigned_payload = old.SerializeToString()
return claim
+61
View File
@@ -0,0 +1,61 @@
import json
import binascii
from google.protobuf import json_format # pylint: disable=no-name-in-module
from lbrynet.schema.error import DecodeError, InvalidAddress
from lbrynet.schema.legacy.migrate import migrate as schema_migrator
from lbrynet.schema.claim import ClaimDict
def migrate_json_claim_value(decoded_json):
try:
if 'fee' in decoded_json:
old_fee = decoded_json['fee']
if not old_fee[list(old_fee.keys())[0]]['amount']:
del decoded_json['fee']
return migrate_json_claim_value(decoded_json)
except (TypeError, AttributeError, InvalidAddress):
raise DecodeError("Failed to decode claim")
try:
pb_migrated = schema_migrator(decoded_json)
return pb_migrated
except json_format.ParseError as parse_error:
raise DecodeError("Failed to parse protobuf: %s" % parse_error)
except Exception as err:
raise DecodeError("Failed to migrate claim: %s" % err)
def smart_decode(claim_value):
"""
Decode a claim value
Try decoding claim protobuf, if this fails try decoding json and migrating it.
If unable to decode or migrate, raise DecodeError
"""
# if already decoded, return
if isinstance(claim_value, ClaimDict):
return claim_value
elif isinstance(claim_value, dict):
return ClaimDict.load_dict(claim_value)
try:
claim_value = binascii.unhexlify(claim_value)
except (TypeError, ValueError):
pass
if claim_value[0] in ['{', ord('{')]:
try:
if isinstance(claim_value, bytes):
claim_value = claim_value.decode()
decoded_json = json.loads(claim_value)
return migrate_json_claim_value(decoded_json)
except (ValueError, TypeError):
pass
try:
if isinstance(claim_value, str):
claim_value = claim_value.encode()
return ClaimDict.deserialize(claim_value)
except (DecodeError, InvalidAddress, KeyError, TypeError):
raise DecodeError(claim_value)
+90
View File
@@ -0,0 +1,90 @@
import base64, binascii
from copy import deepcopy
from lbrynet.schema.address import decode_address, encode_address
from lbrynet.schema.schema import CLAIM_TYPES, CLAIM_TYPE, STREAM_TYPE, CERTIFICATE_TYPE
from lbrynet.schema.schema import SIGNATURE
from lbrynet.schema.error import DecodeError, InvalidAddress
from lbrynet.schema.signature import Signature
def encode_fields(claim_dictionary, detached_signature: Signature):
"""Encode bytes to hex and b58 for return by ClaimDict"""
claim_dictionary = deepcopy(claim_dictionary)
claim_type = CLAIM_TYPES[claim_dictionary[CLAIM_TYPE]]
claim_value = claim_dictionary[claim_type]
if claim_type == CLAIM_TYPES[STREAM_TYPE]:
claim_value['source']['source'] = binascii.hexlify(claim_value['source']['source']).decode()
if 'fee' in claim_value['metadata']:
try:
address = encode_address(claim_value['metadata']['fee']['address'])
except InvalidAddress as err:
raise DecodeError("Invalid fee address: %s" % err)
claim_value['metadata']['fee']['address'] = address
elif claim_type == CLAIM_TYPES[CERTIFICATE_TYPE]:
public_key = claim_value["publicKey"]
claim_value["publicKey"] = binascii.hexlify(public_key).decode()
if SIGNATURE in claim_dictionary:
encoded_sig = binascii.hexlify(claim_dictionary[SIGNATURE]['signature']).decode()
encoded_cert_id = binascii.hexlify(claim_dictionary[SIGNATURE]['certificateId']).decode()
claim_dictionary[SIGNATURE]['signature'] = encoded_sig
claim_dictionary[SIGNATURE]['certificateId'] = encoded_cert_id
elif detached_signature and detached_signature.raw_signature:
claim_dictionary[SIGNATURE] = {
'detached_signature': binascii.hexlify(detached_signature.serialized).decode(),
'certificateId': binascii.hexlify(detached_signature.certificate_id).decode()
}
claim_dictionary[claim_type] = claim_value
return claim_dictionary
def decode_fields(claim_dictionary):
"""Decode hex and b58 encoded bytes in dictionaries given to ClaimDict"""
detached_signature = None
claim_dictionary = deepcopy(claim_dictionary)
claim_type = CLAIM_TYPES[claim_dictionary[CLAIM_TYPE]]
claim_value = claim_dictionary[claim_type]
if claim_type == CLAIM_TYPES[STREAM_TYPE]:
claim_value['source']['source'] = binascii.unhexlify(claim_value['source']['source'])
if 'fee' in claim_value['metadata']:
try:
address = decode_address(claim_value['metadata']['fee']['address'])
except InvalidAddress as err:
raise DecodeError("Invalid fee address: %s" % err)
claim_value['metadata']['fee']['address'] = address
elif claim_type == CLAIM_TYPES[CERTIFICATE_TYPE]:
public_key = binascii.unhexlify(claim_value["publicKey"])
claim_value["publicKey"] = public_key
if SIGNATURE in claim_dictionary and not claim_dictionary[SIGNATURE].get('detached_signature'):
decoded_sig = binascii.unhexlify(claim_dictionary[SIGNATURE]['signature'])
decoded_cert_id = binascii.unhexlify(claim_dictionary[SIGNATURE]['certificateId'])
claim_dictionary[SIGNATURE]['signature'] = decoded_sig
claim_dictionary[SIGNATURE]['certificateId'] = decoded_cert_id
elif claim_dictionary.get(SIGNATURE, {}).get('detached_signature'):
hex_detached_signature = claim_dictionary[SIGNATURE]['detached_signature']
detached_signature = Signature.flagged_parse(binascii.unhexlify(hex_detached_signature))
del claim_dictionary[SIGNATURE]
claim_dictionary[claim_type] = claim_value
return claim_dictionary, detached_signature
def decode_b64_fields(claim_dictionary):
"""Decode b64 encoded bytes in protobuf generated dictionary to be given to ClaimDict"""
claim_dictionary = deepcopy(claim_dictionary)
claim_type = CLAIM_TYPES[claim_dictionary[CLAIM_TYPE]]
claim_value = claim_dictionary[claim_type]
if claim_type == CLAIM_TYPES[STREAM_TYPE]:
claim_value['source']['source'] = base64.b64decode(claim_value['source']['source'])
if 'fee' in claim_value['metadata']:
address = base64.b64decode(claim_value['metadata']['fee']['address'])
claim_value['metadata']['fee']['address'] = address
elif claim_type == CLAIM_TYPES[CERTIFICATE_TYPE]:
public_key = base64.b64decode(claim_value["publicKey"])
claim_value["publicKey"] = public_key
if SIGNATURE in claim_dictionary:
encoded_sig = base64.b64decode(claim_dictionary[SIGNATURE]['signature'])
encoded_cert_id = base64.b64decode(claim_dictionary[SIGNATURE]['certificateId'])
claim_dictionary[SIGNATURE]['signature'] = encoded_sig
claim_dictionary[SIGNATURE]['certificateId'] = encoded_cert_id
claim_dictionary[claim_type] = claim_value
return claim_dictionary
+22
View File
@@ -0,0 +1,22 @@
class UnknownSourceType(Exception):
pass
class InvalidSourceHashLength(Exception):
pass
class DecodeError(Exception):
pass
class URIParseError(Exception):
pass
class CertificateError(Exception):
pass
class InvalidAddress(Exception):
pass
+79
View File
@@ -0,0 +1,79 @@
from collections import OrderedDict
from lbrynet.schema.address import encode_address, decode_address
from lbrynet.schema.schema import CURRENCY_NAMES, CURRENCY_MAP
from lbrynet.schema.schema.fee import Fee as FeeHelper
from lbrynet.schema.proto import fee_pb2
def migrate(fee):
if len(list(fee.keys())) == 3 and 'currency' in fee and 'amount' in fee and 'address' in fee:
return FeeHelper.load({
"version": "_0_0_1",
"currency": fee['currency'],
"amount": fee['amount'],
"address": decode_address(fee['address'])
})
if len(list(fee.keys())) > 1:
raise Exception("Invalid fee")
currency = list(fee.keys())[0]
amount = fee[currency]['amount']
address = fee[currency]['address']
return FeeHelper.load({
"version": "_0_0_1",
"currency": currency,
"amount": amount,
"address": decode_address(address)
})
class Fee(OrderedDict):
def __init__(self, fee):
if (len(fee) == 4 and "version" in fee and "currency" in fee
and "amount" in fee and "address" in fee):
OrderedDict.__init__(self, fee)
else:
OrderedDict.__init__(self, Fee.load_protobuf(migrate(fee)))
@property
def currency(self):
return self['currency']
@property
def address(self):
return self['address']
@property
def amount(self):
return self['amount']
@property
def version(self):
return self['version']
@property
def protobuf(self):
pb = {
"version": self.version,
"currency": CURRENCY_MAP[self.currency],
"address": decode_address(self.address),
"amount": self.amount
}
return FeeHelper.load(pb)
@classmethod
def load_protobuf(cls, pb):
return cls({
"version": pb.version,
"currency": CURRENCY_NAMES[pb.currency],
"address": encode_address(pb.address),
"amount": pb.amount
})
@classmethod
def deserialize(cls, serialized):
pb = fee_pb2.Fee()
pb.ParseFromString(serialized)
return cls.load_protobuf(pb)
+23
View File
@@ -0,0 +1,23 @@
import hashlib
def sha256(x):
if isinstance(x, str):
x = x.encode('utf-8')
return hashlib.sha256(x).digest()
def double_sha256(x):
return sha256(sha256(x))
def ripemd160(x):
if isinstance(x, str):
x = x.encode('utf-8')
md = hashlib.new('ripemd160')
md.update(x)
return md.digest()
def hash160(x):
return ripemd160(sha256(x))
+59
View File
@@ -0,0 +1,59 @@
import jsonschema
class StructuredDict(dict):
"""
A dictionary that enforces a structure specified by a schema, and supports
migration between different versions of the schema.
"""
# To be specified in sub-classes, an array in the format
# [(version, schema, migration), ...]
_versions = []
# Used internally to allow schema lookups by version number
_schemas = {}
version = None
def __init__(self, value, starting_version, migrate=True, target_version=None):
dict.__init__(self, value)
self.version = starting_version
self._schemas = dict([(version, schema) for (version, schema, _) in self._versions])
self.validate(starting_version)
if migrate:
self.migrate(target_version)
def _upgrade_version_range(self, start_version, end_version):
after_starting_version = False
for version, schema, migration in self._versions:
if not after_starting_version:
if version == self.version:
after_starting_version = True
continue
yield version, schema, migration
if end_version and version == end_version:
break
def validate(self, version):
jsonschema.validate(self, self._schemas[version])
def migrate(self, target_version=None):
if target_version:
assert self._versions.index(target_version) > self._versions.index(self.version), \
"Current version is above target version"
for version, schema, migration in self._upgrade_version_range(self.version, target_version):
migration(self)
try:
self.validate(version)
except jsonschema.ValidationError as e:
raise jsonschema.ValidationError(
"Could not migrate to version %s due to validation error: %s" %
(version, e.message))
self.version = version
File renamed without changes.
+276
View File
@@ -0,0 +1,276 @@
VER_001 = {
'$schema': 'http://json-schema.org/draft-04/schema#',
'title': 'LBRY metadata schema 0.0.1',
'definitions': {
'fee_info': {
'type': 'object',
'properties': {
'amount': {
'type': 'number',
'minimum': 0,
'exclusiveMinimum': True,
},
'address': {
'type': 'string'
}
},
}
},
'type': 'object',
'properties': {
'ver': {
'type': 'string',
'default': '0.0.1'
},
'title': {
'type': 'string'
},
'description': {
'type': 'string'
},
'author': {
'type': 'string'
},
'language': {
'type': 'string'
},
'license': {
'type': 'string'
},
'content-type': {
'type': 'string'
},
'sources': {
'type': 'object',
'properties': {
'lbry_sd_hash': {
'type': 'string'
},
'btih': {
'type': 'string'
},
'url': {
'type': 'string'
}
},
'additionalProperties': False
},
'thumbnail': {
'type': 'string'
},
'preview': {
'type': 'string'
},
'fee': {
'properties': {
'LBC': {'$ref': '#/definitions/fee_info'},
'BTC': {'$ref': '#/definitions/fee_info'},
'USD': {'$ref': '#/definitions/fee_info'}
}
},
'contact': {
'type': 'number'
},
'pubkey': {
'type': 'string'
},
},
'required': [
'title', 'description', 'author', 'language', 'license', 'content-type', 'sources'],
'additionalProperties': False
}
VER_002 = {
'$schema': 'http://json-schema.org/draft-04/schema#',
'title': 'LBRY metadata schema 0.0.2',
'definitions': {
'fee_info': {
'type': 'object',
'properties': {
'amount': {
'type': 'number',
'minimum': 0,
'exclusiveMinimum': True,
},
'address': {
'type': 'string'
}
},
}
},
'type': 'object',
'properties': {
'ver': {
'type': 'string',
'enum': ['0.0.2'],
},
'title': {
'type': 'string'
},
'description': {
'type': 'string'
},
'author': {
'type': 'string'
},
'language': {
'type': 'string'
},
'license': {
'type': 'string'
},
'content-type': {
'type': 'string'
},
'sources': {
'type': 'object',
'properties': {
'lbry_sd_hash': {
'type': 'string'
},
'btih': {
'type': 'string'
},
'url': {
'type': 'string'
}
},
'additionalProperties': False
},
'thumbnail': {
'type': 'string'
},
'preview': {
'type': 'string'
},
'fee': {
'properties': {
'LBC': {'$ref': '#/definitions/fee_info'},
'BTC': {'$ref': '#/definitions/fee_info'},
'USD': {'$ref': '#/definitions/fee_info'}
}
},
'contact': {
'type': 'number'
},
'pubkey': {
'type': 'string'
},
'license_url': {
'type': 'string'
},
'nsfw': {
'type': 'boolean',
'default': False
},
},
'required': [
'ver', 'title', 'description', 'author', 'language', 'license',
'content-type', 'sources', 'nsfw'
],
'additionalProperties': False
}
VER_003 = {
'$schema': 'http://json-schema.org/draft-04/schema#',
'title': 'LBRY metadata schema 0.0.3',
'definitions': {
'fee_info': {
'type': 'object',
'properties': {
'amount': {
'type': 'number',
'minimum': 0,
'exclusiveMinimum': True,
},
'address': {
'type': 'string'
}
},
}
},
'type': 'object',
'properties': {
'ver': {
'type': 'string',
'enum': ['0.0.3'],
},
'title': {
'type': 'string'
},
'description': {
'type': 'string'
},
'author': {
'type': 'string'
},
'language': {
'type': 'string'
},
'license': {
'type': 'string'
},
'content_type': {
'type': 'string'
},
'sources': {
'type': 'object',
'properties': {
'lbry_sd_hash': {
'type': 'string'
},
'btih': {
'type': 'string'
},
'url': {
'type': 'string'
}
},
'additionalProperties': False
},
'thumbnail': {
'type': 'string'
},
'preview': {
'type': 'string'
},
'fee': {
'properties': {
'LBC': {'$ref': '#/definitions/fee_info'},
'BTC': {'$ref': '#/definitions/fee_info'},
'USD': {'$ref': '#/definitions/fee_info'}
}
},
'contact': {
'type': 'number'
},
'pubkey': {
'type': 'string'
},
'license_url': {
'type': 'string'
},
'nsfw': {
'type': 'boolean',
'default': False
},
'sig': {
'type': 'string'
}
},
'required': [
'ver', 'title', 'description', 'author', 'language', 'license',
'content_type', 'sources', 'nsfw'
],
'additionalProperties': False,
'dependencies': {
'pubkey': ['sig'],
'sig': ['pubkey']
}
}
+79
View File
@@ -0,0 +1,79 @@
"""
migrate claim json schema (0.0.1-3) to protobuf (0.1.0)
"""
from lbrynet.schema.legacy import metadata_schemas
from lbrynet.schema.claim import ClaimDict
from .StructuredDict import StructuredDict
def migrate_001_to_002(metadata):
metadata['ver'] = '0.0.2'
metadata['nsfw'] = False
def migrate_002_to_003(metadata):
metadata['ver'] = '0.0.3'
if 'content-type' in metadata:
metadata['content_type'] = metadata['content-type']
del metadata['content-type']
class LegacyMetadata(StructuredDict):
current_version = '0.0.3'
_versions = [
('0.0.1', metadata_schemas.VER_001, None),
('0.0.2', metadata_schemas.VER_002, migrate_001_to_002),
('0.0.3', metadata_schemas.VER_003, migrate_002_to_003)
]
def __init__(self, metadata, migrate=True, target_version=None):
if not isinstance(metadata, dict):
raise TypeError("{} is not a dictionary".format(metadata))
starting_version = metadata.get('ver', '0.0.1')
StructuredDict.__init__(self, metadata, starting_version, migrate, target_version)
def migrate_003_to_010(value):
migrated_to_003 = LegacyMetadata(value)
metadata = {
"version": "_0_1_0"
}
for k in ["author", "description", "language", "license", "nsfw", "thumbnail", "title",
"preview"]:
if k in migrated_to_003:
metadata.update({k: migrated_to_003[k]})
if 'license_url' in migrated_to_003:
metadata['licenseUrl'] = migrated_to_003['license_url']
if "fee" in migrated_to_003:
fee = migrated_to_003["fee"]
currency = list(fee.keys())[0]
amount = fee[currency]['amount']
address = fee[currency]['address']
metadata.update(dict(fee={"currency": currency, "version": "_0_0_1",
"amount": amount, "address": address}))
source = {
"source": migrated_to_003['sources']['lbry_sd_hash'],
"contentType": migrated_to_003['content_type'],
"sourceType": "lbry_sd_hash",
"version": "_0_0_1"
}
migrated = {
"version": "_0_0_1",
"claimType": "streamType",
"stream": {
"version": "_0_0_1",
"metadata": metadata,
"source": source
}
}
return ClaimDict.load_dict(migrated)
def migrate(value):
return migrate_003_to_010(value)
-174
View File
@@ -1,174 +0,0 @@
import os
types_map = {
# http://www.iana.org/assignments/media-types
# Type mapping for automated metadata extraction (video, audio, image, document, binary, model)
'.a': ('application/octet-stream', 'binary'),
'.ai': ('application/postscript', 'image'),
'.aif': ('audio/x-aiff', 'audio'),
'.aifc': ('audio/x-aiff', 'audio'),
'.aiff': ('audio/x-aiff', 'audio'),
'.au': ('audio/basic', 'audio'),
'.avi': ('video/x-msvideo', 'video'),
'.bat': ('text/plain', 'document'),
'.bcpio': ('application/x-bcpio', 'binary'),
'.bin': ('application/octet-stream', 'binary'),
'.bmp': ('image/bmp', 'image'),
'.c': ('text/plain', 'document'),
'.cdf': ('application/x-netcdf', 'binary'),
'.cpio': ('application/x-cpio', 'binary'),
'.csh': ('application/x-csh', 'binary'),
'.css': ('text/css', 'document'),
'.csv': ('text/csv', 'document'),
'.dll': ('application/octet-stream', 'binary'),
'.doc': ('application/msword', 'document'),
'.dot': ('application/msword', 'document'),
'.dvi': ('application/x-dvi', 'binary'),
'.eml': ('message/rfc822', 'document'),
'.eps': ('application/postscript', 'document'),
'.epub': ('application/epub+zip', 'document'),
'.etx': ('text/x-setext', 'document'),
'.exe': ('application/octet-stream', 'binary'),
'.gif': ('image/gif', 'image'),
'.gtar': ('application/x-gtar', 'binary'),
'.h': ('text/plain', 'document'),
'.hdf': ('application/x-hdf', 'binary'),
'.htm': ('text/html', 'document'),
'.html': ('text/html', 'document'),
'.ico': ('image/vnd.microsoft.icon', 'image'),
'.ief': ('image/ief', 'image'),
'.iges': ('model/iges', 'model'),
'.jpe': ('image/jpeg', 'image'),
'.jpeg': ('image/jpeg', 'image'),
'.jpg': ('image/jpeg', 'image'),
'.js': ('application/javascript', 'document'),
'.json': ('application/json', 'document'),
'.ksh': ('text/plain', 'document'),
'.latex': ('application/x-latex', 'binary'),
'.m1v': ('video/mpeg', 'video'),
'.m3u': ('application/vnd.apple.mpegurl', 'audio'),
'.m3u8': ('application/vnd.apple.mpegurl', 'audio'),
'.man': ('application/x-troff-man', 'document'),
'.markdown': ('text/markdown', 'document'),
'.md': ('text/markdown', 'document'),
'.me': ('application/x-troff-me', 'binary'),
'.mht': ('message/rfc822', 'document'),
'.mhtml': ('message/rfc822', 'document'),
'.mif': ('application/x-mif', 'binary'),
'.mov': ('video/quicktime', 'video'),
'.movie': ('video/x-sgi-movie', 'video'),
'.mp2': ('audio/mpeg', 'audio'),
'.mp3': ('audio/mpeg', 'audio'),
'.mp4': ('video/mp4', 'video'),
'.mpa': ('video/mpeg', 'video'),
'.mpe': ('video/mpeg', 'video'),
'.mpeg': ('video/mpeg', 'video'),
'.mpg': ('video/mpeg', 'video'),
'.ms': ('application/x-troff-ms', 'binary'),
'.nc': ('application/x-netcdf', 'binary'),
'.nws': ('message/rfc822', 'document'),
'.o': ('application/octet-stream', 'binary'),
'.obj': ('application/octet-stream', 'model'),
'.oda': ('application/oda', 'binary'),
'.p12': ('application/x-pkcs12', 'binary'),
'.p7c': ('application/pkcs7-mime', 'binary'),
'.pbm': ('image/x-portable-bitmap', 'image'),
'.pdf': ('application/pdf', 'document'),
'.pfx': ('application/x-pkcs12', 'binary'),
'.pgm': ('image/x-portable-graymap', 'image'),
'.pl': ('text/plain', 'document'),
'.png': ('image/png', 'image'),
'.pnm': ('image/x-portable-anymap', 'image'),
'.pot': ('application/vnd.ms-powerpoint', 'document'),
'.ppa': ('application/vnd.ms-powerpoint', 'document'),
'.ppm': ('image/x-portable-pixmap', 'image'),
'.pps': ('application/vnd.ms-powerpoint', 'document'),
'.ppt': ('application/vnd.ms-powerpoint', 'document'),
'.ps': ('application/postscript', 'document'),
'.pwz': ('application/vnd.ms-powerpoint', 'document'),
'.py': ('text/x-python', 'document'),
'.pyc': ('application/x-python-code', 'binary'),
'.pyo': ('application/x-python-code', 'binary'),
'.qt': ('video/quicktime', 'video'),
'.ra': ('audio/x-pn-realaudio', 'audio'),
'.ram': ('application/x-pn-realaudio', 'audio'),
'.ras': ('image/x-cmu-raster', 'image'),
'.rdf': ('application/xml', 'binary'),
'.rgb': ('image/x-rgb', 'image'),
'.roff': ('application/x-troff', 'binary'),
'.rtx': ('text/richtext', 'document'),
'.sgm': ('text/x-sgml', 'document'),
'.sgml': ('text/x-sgml', 'document'),
'.sh': ('application/x-sh', 'document'),
'.shar': ('application/x-shar', 'binary'),
'.snd': ('audio/basic', 'audio'),
'.so': ('application/octet-stream', 'binary'),
'.src': ('application/x-wais-source', 'binary'),
'.stl': ('model/stl', 'model'),
'.sv4cpio': ('application/x-sv4cpio', 'binary'),
'.sv4crc': ('application/x-sv4crc', 'binary'),
'.svg': ('image/svg+xml', 'image'),
'.swf': ('application/x-shockwave-flash', 'binary'),
'.t': ('application/x-troff', 'binary'),
'.tar': ('application/x-tar', 'binary'),
'.tcl': ('application/x-tcl', 'binary'),
'.tex': ('application/x-tex', 'binary'),
'.texi': ('application/x-texinfo', 'binary'),
'.texinfo': ('application/x-texinfo', 'binary'),
'.tif': ('image/tiff', 'image'),
'.tiff': ('image/tiff', 'image'),
'.tr': ('application/x-troff', 'binary'),
'.tsv': ('text/tab-separated-values', 'document'),
'.txt': ('text/plain', 'document'),
'.ustar': ('application/x-ustar', 'binary'),
'.vcf': ('text/x-vcard', 'document'),
'.wav': ('audio/x-wav', 'audio'),
'.webm': ('video/webm', 'video'),
'.wiz': ('application/msword', 'document'),
'.wsdl': ('application/xml', 'document'),
'.xbm': ('image/x-xbitmap', 'image'),
'.xlb': ('application/vnd.ms-excel', 'document'),
'.xls': ('application/vnd.ms-excel', 'document'),
'.xml': ('text/xml', 'document'),
'.xpdl': ('application/xml', 'document'),
'.xpm': ('image/x-xpixmap', 'image'),
'.xsl': ('application/xml', 'document'),
'.xwd': ('image/x-xwindowdump', 'image'),
'.zip': ('application/zip', 'binary'),
# These are non-standard types, commonly found in the wild.
'.cbr': ('application/vnd.comicbook-rar', 'document'),
'.cbz': ('application/vnd.comicbook+zip', 'document'),
'.lbry': ('application/x-ext-lbry', 'document'),
'.mid': ('audio/midi', 'audio'),
'.midi': ('audio/midi', 'audio'),
'.mobi': ('application/x-mobipocket-ebook', 'document'),
'.pct': ('image/pict', 'image'),
'.pic': ('image/pict', 'image'),
'.pict': ('image/pict', 'image'),
'.prc': ('application/x-mobipocket-ebook', 'document'),
'.rtf': ('application/rtf', 'document'),
'.xul': ('text/xul', 'document'),
'.m4v': ('video/m4v', 'video'),
# microsoft is special and has its own 'standard'
# https://docs.microsoft.com/en-us/windows/desktop/wmp/file-name-extensions
'.wmv': ('video/x-ms-wmv', 'video')
}
def guess_media_type(path):
_, ext = os.path.splitext(path)
extension = ext.strip().lower()
if extension[1:]:
if extension in types_map:
return types_map[extension]
return f'application/x-ext-{extension[1:]}', 'binary'
return 'application/octet-stream', 'binary'
def guess_stream_type(media_type):
for media, stream in types_map.values():
if media == media_type:
return stream
return 'binary'
File renamed without changes.
+18
View File
@@ -0,0 +1,18 @@
syntax = "proto2";
enum KeyType {
UNKNOWN_PUBLIC_KEY_TYPE = 0;
NIST256p = 1;
NIST384p = 2;
SECP256k1 = 3;
}
message Certificate {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required KeyType keyType = 2;
required bytes publicKey = 4;
}
@@ -1,5 +1,5 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: certificate.proto
# source: lbrynet/schema/proto/certificate.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
@@ -16,16 +16,16 @@ _sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='certificate.proto',
package='legacy_pb',
name='lbrynet/schema/proto/certificate.proto',
package='',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x11\x63\x65rtificate.proto\x12\tlegacy_pb\"\xa2\x01\n\x0b\x43\x65rtificate\x12/\n\x07version\x18\x01 \x02(\x0e\x32\x1e.legacy_pb.Certificate.Version\x12#\n\x07keyType\x18\x02 \x02(\x0e\x32\x12.legacy_pb.KeyType\x12\x11\n\tpublicKey\x18\x04 \x02(\x0c\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01*Q\n\x07KeyType\x12\x1b\n\x17UNKNOWN_PUBLIC_KEY_TYPE\x10\x00\x12\x0c\n\x08NIST256p\x10\x01\x12\x0c\n\x08NIST384p\x10\x02\x12\r\n\tSECP256k1\x10\x03')
serialized_pb=_b('\n&lbrynet/schema/proto/certificate.proto\"\x8e\x01\n\x0b\x43\x65rtificate\x12%\n\x07version\x18\x01 \x02(\x0e\x32\x14.Certificate.Version\x12\x19\n\x07keyType\x18\x02 \x02(\x0e\x32\x08.KeyType\x12\x11\n\tpublicKey\x18\x04 \x02(\x0c\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01*Q\n\x07KeyType\x12\x1b\n\x17UNKNOWN_PUBLIC_KEY_TYPE\x10\x00\x12\x0c\n\x08NIST256p\x10\x01\x12\x0c\n\x08NIST384p\x10\x02\x12\r\n\tSECP256k1\x10\x03')
)
_KEYTYPE = _descriptor.EnumDescriptor(
name='KeyType',
full_name='legacy_pb.KeyType',
full_name='KeyType',
filename=None,
file=DESCRIPTOR,
values=[
@@ -48,8 +48,8 @@ _KEYTYPE = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=197,
serialized_end=278,
serialized_start=187,
serialized_end=268,
)
_sym_db.RegisterEnumDescriptor(_KEYTYPE)
@@ -62,7 +62,7 @@ SECP256k1 = 3
_CERTIFICATE_VERSION = _descriptor.EnumDescriptor(
name='Version',
full_name='legacy_pb.Certificate.Version',
full_name='Certificate.Version',
filename=None,
file=DESCRIPTOR,
values=[
@@ -77,35 +77,35 @@ _CERTIFICATE_VERSION = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=153,
serialized_end=195,
serialized_start=143,
serialized_end=185,
)
_sym_db.RegisterEnumDescriptor(_CERTIFICATE_VERSION)
_CERTIFICATE = _descriptor.Descriptor(
name='Certificate',
full_name='legacy_pb.Certificate',
full_name='Certificate',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='version', full_name='legacy_pb.Certificate.version', index=0,
name='version', full_name='Certificate.version', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='keyType', full_name='legacy_pb.Certificate.keyType', index=1,
name='keyType', full_name='Certificate.keyType', index=1,
number=2, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='publicKey', full_name='legacy_pb.Certificate.publicKey', index=2,
name='publicKey', full_name='Certificate.publicKey', index=2,
number=4, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
@@ -124,8 +124,8 @@ _CERTIFICATE = _descriptor.Descriptor(
extension_ranges=[],
oneofs=[
],
serialized_start=33,
serialized_end=195,
serialized_start=43,
serialized_end=185,
)
_CERTIFICATE.fields_by_name['version'].enum_type = _CERTIFICATE_VERSION
@@ -137,8 +137,8 @@ _sym_db.RegisterFileDescriptor(DESCRIPTOR)
Certificate = _reflection.GeneratedProtocolMessageType('Certificate', (_message.Message,), dict(
DESCRIPTOR = _CERTIFICATE,
__module__ = 'certificate_pb2'
# @@protoc_insertion_point(class_scope:legacy_pb.Certificate)
__module__ = 'lbrynet.schema.proto.certificate_pb2'
# @@protoc_insertion_point(class_scope:Certificate)
))
_sym_db.RegisterMessage(Certificate)
+22
View File
@@ -0,0 +1,22 @@
syntax = "proto2";
import "lbrynet/schema/proto/stream.proto";
import "lbrynet/schema/proto/certificate.proto";
import "lbrynet/schema/proto/signature.proto";
message Claim {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
enum ClaimType {
UNKNOWN_CLAIM_TYPE = 0;
streamType = 1;
certificateType = 2;
}
required ClaimType claimType = 2;
optional Stream stream = 3;
optional Certificate certificate = 4;
optional Signature publisherSignature = 5;
}
@@ -1,5 +1,5 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: legacy_claim.proto
# source: lbrynet/schema/proto/claim.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
@@ -12,25 +12,25 @@ from google.protobuf import symbol_database as _symbol_database
_sym_db = _symbol_database.Default()
from . import stream_pb2 as stream__pb2
from . import certificate_pb2 as certificate__pb2
from . import signature_pb2 as signature__pb2
from lbrynet.schema.proto import stream_pb2 as lbrynet_dot_schema_dot_proto_dot_stream__pb2
from lbrynet.schema.proto import certificate_pb2 as lbrynet_dot_schema_dot_proto_dot_certificate__pb2
from lbrynet.schema.proto import signature_pb2 as lbrynet_dot_schema_dot_proto_dot_signature__pb2
DESCRIPTOR = _descriptor.FileDescriptor(
name='legacy_claim.proto',
package='legacy_pb',
name='lbrynet/schema/proto/claim.proto',
package='',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x12legacy_claim.proto\x12\tlegacy_pb\x1a\x0cstream.proto\x1a\x11\x63\x65rtificate.proto\x1a\x0fsignature.proto\"\xd9\x02\n\x05\x43laim\x12)\n\x07version\x18\x01 \x02(\x0e\x32\x18.legacy_pb.Claim.Version\x12-\n\tclaimType\x18\x02 \x02(\x0e\x32\x1a.legacy_pb.Claim.ClaimType\x12!\n\x06stream\x18\x03 \x01(\x0b\x32\x11.legacy_pb.Stream\x12+\n\x0b\x63\x65rtificate\x18\x04 \x01(\x0b\x32\x16.legacy_pb.Certificate\x12\x30\n\x12publisherSignature\x18\x05 \x01(\x0b\x32\x14.legacy_pb.Signature\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\"H\n\tClaimType\x12\x16\n\x12UNKNOWN_CLAIM_TYPE\x10\x00\x12\x0e\n\nstreamType\x10\x01\x12\x13\n\x0f\x63\x65rtificateType\x10\x02')
serialized_pb=_b('\n lbrynet/schema/proto/claim.proto\x1a!lbrynet/schema/proto/stream.proto\x1a&lbrynet/schema/proto/certificate.proto\x1a$lbrynet/schema/proto/signature.proto\"\xa7\x02\n\x05\x43laim\x12\x1f\n\x07version\x18\x01 \x02(\x0e\x32\x0e.Claim.Version\x12#\n\tclaimType\x18\x02 \x02(\x0e\x32\x10.Claim.ClaimType\x12\x17\n\x06stream\x18\x03 \x01(\x0b\x32\x07.Stream\x12!\n\x0b\x63\x65rtificate\x18\x04 \x01(\x0b\x32\x0c.Certificate\x12&\n\x12publisherSignature\x18\x05 \x01(\x0b\x32\n.Signature\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\"H\n\tClaimType\x12\x16\n\x12UNKNOWN_CLAIM_TYPE\x10\x00\x12\x0e\n\nstreamType\x10\x01\x12\x13\n\x0f\x63\x65rtificateType\x10\x02')
,
dependencies=[stream__pb2.DESCRIPTOR,certificate__pb2.DESCRIPTOR,signature__pb2.DESCRIPTOR,])
dependencies=[lbrynet_dot_schema_dot_proto_dot_stream__pb2.DESCRIPTOR,lbrynet_dot_schema_dot_proto_dot_certificate__pb2.DESCRIPTOR,lbrynet_dot_schema_dot_proto_dot_signature__pb2.DESCRIPTOR,])
_CLAIM_VERSION = _descriptor.EnumDescriptor(
name='Version',
full_name='legacy_pb.Claim.Version',
full_name='Claim.Version',
filename=None,
file=DESCRIPTOR,
values=[
@@ -45,14 +45,14 @@ _CLAIM_VERSION = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=313,
serialized_end=355,
serialized_start=329,
serialized_end=371,
)
_sym_db.RegisterEnumDescriptor(_CLAIM_VERSION)
_CLAIM_CLAIMTYPE = _descriptor.EnumDescriptor(
name='ClaimType',
full_name='legacy_pb.Claim.ClaimType',
full_name='Claim.ClaimType',
filename=None,
file=DESCRIPTOR,
values=[
@@ -71,49 +71,49 @@ _CLAIM_CLAIMTYPE = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=357,
serialized_end=429,
serialized_start=373,
serialized_end=445,
)
_sym_db.RegisterEnumDescriptor(_CLAIM_CLAIMTYPE)
_CLAIM = _descriptor.Descriptor(
name='Claim',
full_name='legacy_pb.Claim',
full_name='Claim',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='version', full_name='legacy_pb.Claim.version', index=0,
name='version', full_name='Claim.version', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='claimType', full_name='legacy_pb.Claim.claimType', index=1,
name='claimType', full_name='Claim.claimType', index=1,
number=2, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='stream', full_name='legacy_pb.Claim.stream', index=2,
name='stream', full_name='Claim.stream', index=2,
number=3, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='certificate', full_name='legacy_pb.Claim.certificate', index=3,
name='certificate', full_name='Claim.certificate', index=3,
number=4, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='publisherSignature', full_name='legacy_pb.Claim.publisherSignature', index=4,
name='publisherSignature', full_name='Claim.publisherSignature', index=4,
number=5, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
@@ -133,15 +133,15 @@ _CLAIM = _descriptor.Descriptor(
extension_ranges=[],
oneofs=[
],
serialized_start=84,
serialized_end=429,
serialized_start=150,
serialized_end=445,
)
_CLAIM.fields_by_name['version'].enum_type = _CLAIM_VERSION
_CLAIM.fields_by_name['claimType'].enum_type = _CLAIM_CLAIMTYPE
_CLAIM.fields_by_name['stream'].message_type = stream__pb2._STREAM
_CLAIM.fields_by_name['certificate'].message_type = certificate__pb2._CERTIFICATE
_CLAIM.fields_by_name['publisherSignature'].message_type = signature__pb2._SIGNATURE
_CLAIM.fields_by_name['stream'].message_type = lbrynet_dot_schema_dot_proto_dot_stream__pb2._STREAM
_CLAIM.fields_by_name['certificate'].message_type = lbrynet_dot_schema_dot_proto_dot_certificate__pb2._CERTIFICATE
_CLAIM.fields_by_name['publisherSignature'].message_type = lbrynet_dot_schema_dot_proto_dot_signature__pb2._SIGNATURE
_CLAIM_VERSION.containing_type = _CLAIM
_CLAIM_CLAIMTYPE.containing_type = _CLAIM
DESCRIPTOR.message_types_by_name['Claim'] = _CLAIM
@@ -149,8 +149,8 @@ _sym_db.RegisterFileDescriptor(DESCRIPTOR)
Claim = _reflection.GeneratedProtocolMessageType('Claim', (_message.Message,), dict(
DESCRIPTOR = _CLAIM,
__module__ = 'legacy_claim_pb2'
# @@protoc_insertion_point(class_scope:legacy_pb.Claim)
__module__ = 'lbrynet.schema.proto.claim_pb2'
# @@protoc_insertion_point(class_scope:Claim)
))
_sym_db.RegisterMessage(Claim)
+18
View File
@@ -0,0 +1,18 @@
syntax = "proto2";
message Fee {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
enum Currency {
UNKNOWN_CURRENCY = 0;
LBC = 1;
BTC = 2;
USD = 3;
}
required Version version = 1;
required Currency currency = 2;
required bytes address = 3;
required float amount = 4;
}
@@ -1,5 +1,5 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: fee.proto
# source: lbrynet/schema/proto/fee.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
@@ -15,18 +15,18 @@ _sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='fee.proto',
package='legacy_pb',
name='lbrynet/schema/proto/fee.proto',
package='',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\tfee.proto\x12\tlegacy_pb\"\xe3\x01\n\x03\x46\x65\x65\x12\'\n\x07version\x18\x01 \x02(\x0e\x32\x16.legacy_pb.Fee.Version\x12)\n\x08\x63urrency\x18\x02 \x02(\x0e\x32\x17.legacy_pb.Fee.Currency\x12\x0f\n\x07\x61\x64\x64ress\x18\x03 \x02(\x0c\x12\x0e\n\x06\x61mount\x18\x04 \x02(\x02\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\";\n\x08\x43urrency\x12\x14\n\x10UNKNOWN_CURRENCY\x10\x00\x12\x07\n\x03LBC\x10\x01\x12\x07\n\x03\x42TC\x10\x02\x12\x07\n\x03USD\x10\x03')
serialized_pb=_b('\n\x1elbrynet/schema/proto/fee.proto\"\xcf\x01\n\x03\x46\x65\x65\x12\x1d\n\x07version\x18\x01 \x02(\x0e\x32\x0c.Fee.Version\x12\x1f\n\x08\x63urrency\x18\x02 \x02(\x0e\x32\r.Fee.Currency\x12\x0f\n\x07\x61\x64\x64ress\x18\x03 \x02(\x0c\x12\x0e\n\x06\x61mount\x18\x04 \x02(\x02\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\";\n\x08\x43urrency\x12\x14\n\x10UNKNOWN_CURRENCY\x10\x00\x12\x07\n\x03LBC\x10\x01\x12\x07\n\x03\x42TC\x10\x02\x12\x07\n\x03USD\x10\x03')
)
_FEE_VERSION = _descriptor.EnumDescriptor(
name='Version',
full_name='legacy_pb.Fee.Version',
full_name='Fee.Version',
filename=None,
file=DESCRIPTOR,
values=[
@@ -41,14 +41,14 @@ _FEE_VERSION = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=149,
serialized_end=191,
serialized_start=139,
serialized_end=181,
)
_sym_db.RegisterEnumDescriptor(_FEE_VERSION)
_FEE_CURRENCY = _descriptor.EnumDescriptor(
name='Currency',
full_name='legacy_pb.Fee.Currency',
full_name='Fee.Currency',
filename=None,
file=DESCRIPTOR,
values=[
@@ -71,42 +71,42 @@ _FEE_CURRENCY = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=193,
serialized_end=252,
serialized_start=183,
serialized_end=242,
)
_sym_db.RegisterEnumDescriptor(_FEE_CURRENCY)
_FEE = _descriptor.Descriptor(
name='Fee',
full_name='legacy_pb.Fee',
full_name='Fee',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='version', full_name='legacy_pb.Fee.version', index=0,
name='version', full_name='Fee.version', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='currency', full_name='legacy_pb.Fee.currency', index=1,
name='currency', full_name='Fee.currency', index=1,
number=2, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='address', full_name='legacy_pb.Fee.address', index=2,
name='address', full_name='Fee.address', index=2,
number=3, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='amount', full_name='legacy_pb.Fee.amount', index=3,
name='amount', full_name='Fee.amount', index=3,
number=4, type=2, cpp_type=6, label=2,
has_default_value=False, default_value=float(0),
message_type=None, enum_type=None, containing_type=None,
@@ -126,8 +126,8 @@ _FEE = _descriptor.Descriptor(
extension_ranges=[],
oneofs=[
],
serialized_start=25,
serialized_end=252,
serialized_start=35,
serialized_end=242,
)
_FEE.fields_by_name['version'].enum_type = _FEE_VERSION
@@ -139,8 +139,8 @@ _sym_db.RegisterFileDescriptor(DESCRIPTOR)
Fee = _reflection.GeneratedProtocolMessageType('Fee', (_message.Message,), dict(
DESCRIPTOR = _FEE,
__module__ = 'fee_pb2'
# @@protoc_insertion_point(class_scope:legacy_pb.Fee)
__module__ = 'lbrynet.schema.proto.fee_pb2'
# @@protoc_insertion_point(class_scope:Fee)
))
_sym_db.RegisterMessage(Fee)
+212
View File
@@ -0,0 +1,212 @@
syntax = "proto2";
import "lbrynet/schema/proto/fee.proto";
message Metadata {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
_0_0_2 = 2;
_0_0_3 = 3;
_0_1_0 = 4;
}
enum Language {
UNKNOWN_LANGUAGE = 0;
en = 1;
aa = 2;
ab = 3;
ae = 4;
af = 5;
ak = 6;
am = 7;
an = 8;
ar = 9;
as = 10;
av = 11;
ay = 12;
az = 13;
ba = 14;
be = 15;
bg = 16;
bh = 17;
bi = 18;
bm = 19;
bn = 20;
bo = 21;
br = 22;
bs = 23;
ca = 24;
ce = 25;
ch = 26;
co = 27;
cr = 28;
cs = 29;
cu = 30;
cv = 31;
cy = 32;
da = 33;
de = 34;
dv = 35;
dz = 36;
ee = 37;
el = 38;
eo = 39;
es = 40;
et = 41;
eu = 42;
fa = 43;
ff = 44;
fi = 45;
fj = 46;
fo = 47;
fr = 48;
fy = 49;
ga = 50;
gd = 51;
gl = 52;
gn = 53;
gu = 54;
gv = 55;
ha = 56;
he = 57;
hi = 58;
ho = 59;
hr = 60;
ht = 61;
hu = 62;
hy = 63;
hz = 64;
ia = 65;
id = 66;
ie = 67;
ig = 68;
ii = 69;
ik = 70;
io = 71;
is = 72;
it = 73;
iu = 74;
ja = 75;
jv = 76;
ka = 77;
kg = 78;
ki = 79;
kj = 80;
kk = 81;
kl = 82;
km = 83;
kn = 84;
ko = 85;
kr = 86;
ks = 87;
ku = 88;
kv = 89;
kw = 90;
ky = 91;
la = 92;
lb = 93;
lg = 94;
li = 95;
ln = 96;
lo = 97;
lt = 98;
lu = 99;
lv = 100;
mg = 101;
mh = 102;
mi = 103;
mk = 104;
ml = 105;
mn = 106;
mr = 107;
ms = 108;
mt = 109;
my = 110;
na = 111;
nb = 112;
nd = 113;
ne = 114;
ng = 115;
nl = 116;
nn = 117;
no = 118;
nr = 119;
nv = 120;
ny = 121;
oc = 122;
oj = 123;
om = 124;
or = 125;
os = 126;
pa = 127;
pi = 128;
pl = 129;
ps = 130;
pt = 131;
qu = 132;
rm = 133;
rn = 134;
ro = 135;
ru = 136;
rw = 137;
sa = 138;
sc = 139;
sd = 140;
se = 141;
sg = 142;
si = 143;
sk = 144;
sl = 145;
sm = 146;
sn = 147;
so = 148;
sq = 149;
sr = 150;
ss = 151;
st = 152;
su = 153;
sv = 154;
sw = 155;
ta = 156;
te = 157;
tg = 158;
th = 159;
ti = 160;
tk = 161;
tl = 162;
tn = 163;
to = 164;
tr = 165;
ts = 166;
tt = 167;
tw = 168;
ty = 169;
ug = 170;
uk = 171;
ur = 172;
uz = 173;
ve = 174;
vi = 175;
vo = 176;
wa = 177;
wo = 178;
xh = 179;
yi = 180;
yo = 181;
za = 182;
zh = 183;
zu = 184;
}
required Version version = 1;
required Language language = 2;
required string title = 3;
required string description = 4;
required string author = 5;
required string license = 6;
required bool nsfw = 7;
optional Fee fee = 8;
optional string thumbnail = 9;
optional string preview = 10;
optional string licenseUrl = 11;
}
@@ -1,5 +1,5 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: metadata.proto
# source: lbrynet/schema/proto/metadata.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
@@ -12,23 +12,23 @@ from google.protobuf import symbol_database as _symbol_database
_sym_db = _symbol_database.Default()
from . import fee_pb2 as fee__pb2
from lbrynet.schema.proto import fee_pb2 as lbrynet_dot_schema_dot_proto_dot_fee__pb2
DESCRIPTOR = _descriptor.FileDescriptor(
name='metadata.proto',
package='legacy_pb',
name='lbrynet/schema/proto/metadata.proto',
package='',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x0emetadata.proto\x12\tlegacy_pb\x1a\tfee.proto\"\xfc\x0e\n\x08Metadata\x12,\n\x07version\x18\x01 \x02(\x0e\x32\x1b.legacy_pb.Metadata.Version\x12.\n\x08language\x18\x02 \x02(\x0e\x32\x1c.legacy_pb.Metadata.Language\x12\r\n\x05title\x18\x03 \x02(\t\x12\x13\n\x0b\x64\x65scription\x18\x04 \x02(\t\x12\x0e\n\x06\x61uthor\x18\x05 \x02(\t\x12\x0f\n\x07license\x18\x06 \x02(\t\x12\x0c\n\x04nsfw\x18\x07 \x02(\x08\x12\x1b\n\x03\x66\x65\x65\x18\x08 \x01(\x0b\x32\x0e.legacy_pb.Fee\x12\x11\n\tthumbnail\x18\t \x01(\t\x12\x0f\n\x07preview\x18\n \x01(\t\x12\x12\n\nlicenseUrl\x18\x0b \x01(\t\"N\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\x12\n\n\x06_0_0_2\x10\x02\x12\n\n\x06_0_0_3\x10\x03\x12\n\n\x06_0_1_0\x10\x04\"\x99\x0c\n\x08Language\x12\x14\n\x10UNKNOWN_LANGUAGE\x10\x00\x12\x06\n\x02\x65n\x10\x01\x12\x06\n\x02\x61\x61\x10\x02\x12\x06\n\x02\x61\x62\x10\x03\x12\x06\n\x02\x61\x65\x10\x04\x12\x06\n\x02\x61\x66\x10\x05\x12\x06\n\x02\x61k\x10\x06\x12\x06\n\x02\x61m\x10\x07\x12\x06\n\x02\x61n\x10\x08\x12\x06\n\x02\x61r\x10\t\x12\x06\n\x02\x61s\x10\n\x12\x06\n\x02\x61v\x10\x0b\x12\x06\n\x02\x61y\x10\x0c\x12\x06\n\x02\x61z\x10\r\x12\x06\n\x02\x62\x61\x10\x0e\x12\x06\n\x02\x62\x65\x10\x0f\x12\x06\n\x02\x62g\x10\x10\x12\x06\n\x02\x62h\x10\x11\x12\x06\n\x02\x62i\x10\x12\x12\x06\n\x02\x62m\x10\x13\x12\x06\n\x02\x62n\x10\x14\x12\x06\n\x02\x62o\x10\x15\x12\x06\n\x02\x62r\x10\x16\x12\x06\n\x02\x62s\x10\x17\x12\x06\n\x02\x63\x61\x10\x18\x12\x06\n\x02\x63\x65\x10\x19\x12\x06\n\x02\x63h\x10\x1a\x12\x06\n\x02\x63o\x10\x1b\x12\x06\n\x02\x63r\x10\x1c\x12\x06\n\x02\x63s\x10\x1d\x12\x06\n\x02\x63u\x10\x1e\x12\x06\n\x02\x63v\x10\x1f\x12\x06\n\x02\x63y\x10 \x12\x06\n\x02\x64\x61\x10!\x12\x06\n\x02\x64\x65\x10\"\x12\x06\n\x02\x64v\x10#\x12\x06\n\x02\x64z\x10$\x12\x06\n\x02\x65\x65\x10%\x12\x06\n\x02\x65l\x10&\x12\x06\n\x02\x65o\x10\'\x12\x06\n\x02\x65s\x10(\x12\x06\n\x02\x65t\x10)\x12\x06\n\x02\x65u\x10*\x12\x06\n\x02\x66\x61\x10+\x12\x06\n\x02\x66\x66\x10,\x12\x06\n\x02\x66i\x10-\x12\x06\n\x02\x66j\x10.\x12\x06\n\x02\x66o\x10/\x12\x06\n\x02\x66r\x10\x30\x12\x06\n\x02\x66y\x10\x31\x12\x06\n\x02ga\x10\x32\x12\x06\n\x02gd\x10\x33\x12\x06\n\x02gl\x10\x34\x12\x06\n\x02gn\x10\x35\x12\x06\n\x02gu\x10\x36\x12\x06\n\x02gv\x10\x37\x12\x06\n\x02ha\x10\x38\x12\x06\n\x02he\x10\x39\x12\x06\n\x02hi\x10:\x12\x06\n\x02ho\x10;\x12\x06\n\x02hr\x10<\x12\x06\n\x02ht\x10=\x12\x06\n\x02hu\x10>\x12\x06\n\x02hy\x10?\x12\x06\n\x02hz\x10@\x12\x06\n\x02ia\x10\x41\x12\x06\n\x02id\x10\x42\x12\x06\n\x02ie\x10\x43\x12\x06\n\x02ig\x10\x44\x12\x06\n\x02ii\x10\x45\x12\x06\n\x02ik\x10\x46\x12\x06\n\x02io\x10G\x12\x06\n\x02is\x10H\x12\x06\n\x02it\x10I\x12\x06\n\x02iu\x10J\x12\x06\n\x02ja\x10K\x12\x06\n\x02jv\x10L\x12\x06\n\x02ka\x10M\x12\x06\n\x02kg\x10N\x12\x06\n\x02ki\x10O\x12\x06\n\x02kj\x10P\x12\x06\n\x02kk\x10Q\x12\x06\n\x02kl\x10R\x12\x06\n\x02km\x10S\x12\x06\n\x02kn\x10T\x12\x06\n\x02ko\x10U\x12\x06\n\x02kr\x10V\x12\x06\n\x02ks\x10W\x12\x06\n\x02ku\x10X\x12\x06\n\x02kv\x10Y\x12\x06\n\x02kw\x10Z\x12\x06\n\x02ky\x10[\x12\x06\n\x02la\x10\\\x12\x06\n\x02lb\x10]\x12\x06\n\x02lg\x10^\x12\x06\n\x02li\x10_\x12\x06\n\x02ln\x10`\x12\x06\n\x02lo\x10\x61\x12\x06\n\x02lt\x10\x62\x12\x06\n\x02lu\x10\x63\x12\x06\n\x02lv\x10\x64\x12\x06\n\x02mg\x10\x65\x12\x06\n\x02mh\x10\x66\x12\x06\n\x02mi\x10g\x12\x06\n\x02mk\x10h\x12\x06\n\x02ml\x10i\x12\x06\n\x02mn\x10j\x12\x06\n\x02mr\x10k\x12\x06\n\x02ms\x10l\x12\x06\n\x02mt\x10m\x12\x06\n\x02my\x10n\x12\x06\n\x02na\x10o\x12\x06\n\x02nb\x10p\x12\x06\n\x02nd\x10q\x12\x06\n\x02ne\x10r\x12\x06\n\x02ng\x10s\x12\x06\n\x02nl\x10t\x12\x06\n\x02nn\x10u\x12\x06\n\x02no\x10v\x12\x06\n\x02nr\x10w\x12\x06\n\x02nv\x10x\x12\x06\n\x02ny\x10y\x12\x06\n\x02oc\x10z\x12\x06\n\x02oj\x10{\x12\x06\n\x02om\x10|\x12\x06\n\x02or\x10}\x12\x06\n\x02os\x10~\x12\x06\n\x02pa\x10\x7f\x12\x07\n\x02pi\x10\x80\x01\x12\x07\n\x02pl\x10\x81\x01\x12\x07\n\x02ps\x10\x82\x01\x12\x07\n\x02pt\x10\x83\x01\x12\x07\n\x02qu\x10\x84\x01\x12\x07\n\x02rm\x10\x85\x01\x12\x07\n\x02rn\x10\x86\x01\x12\x07\n\x02ro\x10\x87\x01\x12\x07\n\x02ru\x10\x88\x01\x12\x07\n\x02rw\x10\x89\x01\x12\x07\n\x02sa\x10\x8a\x01\x12\x07\n\x02sc\x10\x8b\x01\x12\x07\n\x02sd\x10\x8c\x01\x12\x07\n\x02se\x10\x8d\x01\x12\x07\n\x02sg\x10\x8e\x01\x12\x07\n\x02si\x10\x8f\x01\x12\x07\n\x02sk\x10\x90\x01\x12\x07\n\x02sl\x10\x91\x01\x12\x07\n\x02sm\x10\x92\x01\x12\x07\n\x02sn\x10\x93\x01\x12\x07\n\x02so\x10\x94\x01\x12\x07\n\x02sq\x10\x95\x01\x12\x07\n\x02sr\x10\x96\x01\x12\x07\n\x02ss\x10\x97\x01\x12\x07\n\x02st\x10\x98\x01\x12\x07\n\x02su\x10\x99\x01\x12\x07\n\x02sv\x10\x9a\x01\x12\x07\n\x02sw\x10\x9b\x01\x12\x07\n\x02ta\x10\x9c\x01\x12\x07\n\x02te\x10\x9d\x01\x12\x07\n\x02tg\x10\x9e\x01\x12\x07\n\x02th\x10\x9f\x01\x12\x07\n\x02ti\x10\xa0\x01\x12\x07\n\x02tk\x10\xa1\x01\x12\x07\n\x02tl\x10\xa2\x01\x12\x07\n\x02tn\x10\xa3\x01\x12\x07\n\x02to\x10\xa4\x01\x12\x07\n\x02tr\x10\xa5\x01\x12\x07\n\x02ts\x10\xa6\x01\x12\x07\n\x02tt\x10\xa7\x01\x12\x07\n\x02tw\x10\xa8\x01\x12\x07\n\x02tyLine truncated
serialized_pb=_b('\n#lbrynet/schema/proto/metadata.proto\x1a\x1elbrynet/schema/proto/fee.proto\"\xde\x0e\n\x08Metadata\x12\"\n\x07version\x18\x01 \x02(\x0e\x32\x11.Metadata.Version\x12$\n\x08language\x18\x02 \x02(\x0e\x32\x12.Metadata.Language\x12\r\n\x05title\x18\x03 \x02(\t\x12\x13\n\x0b\x64\x65scription\x18\x04 \x02(\t\x12\x0e\n\x06\x61uthor\x18\x05 \x02(\t\x12\x0f\n\x07license\x18\x06 \x02(\t\x12\x0c\n\x04nsfw\x18\x07 \x02(\x08\x12\x11\n\x03\x66\x65\x65\x18\x08 \x01(\x0b\x32\x04.Fee\x12\x11\n\tthumbnail\x18\t \x01(\t\x12\x0f\n\x07preview\x18\n \x01(\t\x12\x12\n\nlicenseUrl\x18\x0b \x01(\t\"N\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\x12\n\n\x06_0_0_2\x10\x02\x12\n\n\x06_0_0_3\x10\x03\x12\n\n\x06_0_1_0\x10\x04\"\x99\x0c\n\x08Language\x12\x14\n\x10UNKNOWN_LANGUAGE\x10\x00\x12\x06\n\x02\x65n\x10\x01\x12\x06\n\x02\x61\x61\x10\x02\x12\x06\n\x02\x61\x62\x10\x03\x12\x06\n\x02\x61\x65\x10\x04\x12\x06\n\x02\x61\x66\x10\x05\x12\x06\n\x02\x61k\x10\x06\x12\x06\n\x02\x61m\x10\x07\x12\x06\n\x02\x61n\x10\x08\x12\x06\n\x02\x61r\x10\t\x12\x06\n\x02\x61s\x10\n\x12\x06\n\x02\x61v\x10\x0b\x12\x06\n\x02\x61y\x10\x0c\x12\x06\n\x02\x61z\x10\r\x12\x06\n\x02\x62\x61\x10\x0e\x12\x06\n\x02\x62\x65\x10\x0f\x12\x06\n\x02\x62g\x10\x10\x12\x06\n\x02\x62h\x10\x11\x12\x06\n\x02\x62i\x10\x12\x12\x06\n\x02\x62m\x10\x13\x12\x06\n\x02\x62n\x10\x14\x12\x06\n\x02\x62o\x10\x15\x12\x06\n\x02\x62r\x10\x16\x12\x06\n\x02\x62s\x10\x17\x12\x06\n\x02\x63\x61\x10\x18\x12\x06\n\x02\x63\x65\x10\x19\x12\x06\n\x02\x63h\x10\x1a\x12\x06\n\x02\x63o\x10\x1b\x12\x06\n\x02\x63r\x10\x1c\x12\x06\n\x02\x63s\x10\x1d\x12\x06\n\x02\x63u\x10\x1e\x12\x06\n\x02\x63v\x10\x1f\x12\x06\n\x02\x63y\x10 \x12\x06\n\x02\x64\x61\x10!\x12\x06\n\x02\x64\x65\x10\"\x12\x06\n\x02\x64v\x10#\x12\x06\n\x02\x64z\x10$\x12\x06\n\x02\x65\x65\x10%\x12\x06\n\x02\x65l\x10&\x12\x06\n\x02\x65o\x10\'\x12\x06\n\x02\x65s\x10(\x12\x06\n\x02\x65t\x10)\x12\x06\n\x02\x65u\x10*\x12\x06\n\x02\x66\x61\x10+\x12\x06\n\x02\x66\x66\x10,\x12\x06\n\x02\x66i\x10-\x12\x06\n\x02\x66j\x10.\x12\x06\n\x02\x66o\x10/\x12\x06\n\x02\x66r\x10\x30\x12\x06\n\x02\x66y\x10\x31\x12\x06\n\x02ga\x10\x32\x12\x06\n\x02gd\x10\x33\x12\x06\n\x02gl\x10\x34\x12\x06\n\x02gn\x10\x35\x12\x06\n\x02gu\x10\x36\x12\x06\n\x02gv\x10\x37\x12\x06\n\x02ha\x10\x38\x12\x06\n\x02he\x10\x39\x12\x06\n\x02hi\x10:\x12\x06\n\x02ho\x10;\x12\x06\n\x02hr\x10<\x12\x06\n\x02ht\x10=\x12\x06\n\x02hu\x10>\x12\x06\n\x02hy\x10?\x12\x06\n\x02hz\x10@\x12\x06\n\x02ia\x10\x41\x12\x06\n\x02id\x10\x42\x12\x06\n\x02ie\x10\x43\x12\x06\n\x02ig\x10\x44\x12\x06\n\x02ii\x10\x45\x12\x06\n\x02ik\x10\x46\x12\x06\n\x02io\x10G\x12\x06\n\x02is\x10H\x12\x06\n\x02it\x10I\x12\x06\n\x02iu\x10J\x12\x06\n\x02ja\x10K\x12\x06\n\x02jv\x10L\x12\x06\n\x02ka\x10M\x12\x06\n\x02kg\x10N\x12\x06\n\x02ki\x10O\x12\x06\n\x02kj\x10P\x12\x06\n\x02kk\x10Q\x12\x06\n\x02kl\x10R\x12\x06\n\x02km\x10S\x12\x06\n\x02kn\x10T\x12\x06\n\x02ko\x10U\x12\x06\n\x02kr\x10V\x12\x06\n\x02ks\x10W\x12\x06\n\x02ku\x10X\x12\x06\n\x02kv\x10Y\x12\x06\n\x02kw\x10Z\x12\x06\n\x02ky\x10[\x12\x06\n\x02la\x10\\\x12\x06\n\x02lb\x10]\x12\x06\n\x02lg\x10^\x12\x06\n\x02li\x10_\x12\x06\n\x02ln\x10`\x12\x06\n\x02lo\x10\x61\x12\x06\n\x02lt\x10\x62\x12\x06\n\x02lu\x10\x63\x12\x06\n\x02lv\x10\x64\x12\x06\n\x02mg\x10\x65\x12\x06\n\x02mh\x10\x66\x12\x06\n\x02mi\x10g\x12\x06\n\x02mk\x10h\x12\x06\n\x02ml\x10i\x12\x06\n\x02mn\x10j\x12\x06\n\x02mr\x10k\x12\x06\n\x02ms\x10l\x12\x06\n\x02mt\x10m\x12\x06\n\x02my\x10n\x12\x06\n\x02na\x10o\x12\x06\n\x02nb\x10p\x12\x06\n\x02nd\x10q\x12\x06\n\x02ne\x10r\x12\x06\n\x02ng\x10s\x12\x06\n\x02nl\x10t\x12\x06\n\x02nn\x10u\x12\x06\n\x02no\x10v\x12\x06\n\x02nr\x10w\x12\x06\n\x02nv\x10x\x12\x06\n\x02ny\x10y\x12\x06\n\x02oc\x10z\x12\x06\n\x02oj\x10{\x12\x06\n\x02om\x10|\x12\x06\n\x02or\x10}\x12\x06\n\x02os\x10~\x12\x06\n\x02pa\x10\x7f\x12\x07\n\x02pi\x10\x80\x01\x12\x07\n\x02pl\x10\x81\x01\x12\x07\n\x02ps\x10\x82\x01\x12\x07\n\x02pt\x10\x83\x01\x12\x07\n\x02qu\x10\x84\x01\x12\x07\n\x02rm\x10\x85\x01\x12\x07\n\x02rn\x10\x86\x01\x12\x07\n\x02ro\x10\x87\x01\x12\x07\n\x02ru\x10\x88\x01\x12\x07\n\x02rw\x10\x89\x01\x12\x07\n\x02sa\x10\x8a\x01\x12\x07\n\x02sc\x10\x8b\x01\x12\x07\n\x02sd\x10\x8c\x01\x12\x07\n\x02se\x10\x8d\x01\x12\x07\n\x02sg\x10\x8e\x01\x12\x07\n\x02si\x10\x8f\x01\x12\x07\n\x02sk\x10\x90\x01\x12\x07\n\x02sl\x10\x91\x01\x12\x07\n\x02sm\x10\x92\x01\x12\x07\n\x02sn\x10\x93\x01\x12\x07\n\x02so\x10\x94\x01\x12\x07\n\x02sq\x10\x95\x01\x12\x07\n\x02sr\x10\x96\x01\x12\x07\n\x02ss\x10\x97\x01\x12\x07\n\x02st\x10\x98\x01\x12\x07\n\x02su\x10\x99\x01\x12\x07\n\x02sv\x10\x9a\x01\x12\x07\n\x02sw\x10\x9b\x01\x12\x07\n\x02ta\x10\x9c\x01\x12\x07\n\x02te\x10\x9d\x01\x12\x07\n\x02tg\x10\x9e\x01\x12\x07\n\x02th\x10\x9f\x01\x12\x07\n\x02ti\x10\xa0\x01\x12\x07\n\x02tk\x10\xa1\x01\x12\x07\n\x02tl\x10\xa2\x01\x12\x07\n\x02tn\x10\xa3\x01\x12\x07\n\x02to\x10\xa4\x01\x12\x07\n\x02tr\x10\xa5\x01\x12\x07\n\x02ts\x10\xa6\x01\x12\x07\n\x02tt\x10\xa7\x01\x12\x07\n\x02tw\x10\xa8\x01\x12\x07\n\x02ty\x1Line truncated
,
dependencies=[fee__pb2.DESCRIPTOR,])
dependencies=[lbrynet_dot_schema_dot_proto_dot_fee__pb2.DESCRIPTOR,])
_METADATA_VERSION = _descriptor.EnumDescriptor(
name='Version',
full_name='legacy_pb.Metadata.Version',
full_name='Metadata.Version',
filename=None,
file=DESCRIPTOR,
values=[
@@ -55,14 +55,14 @@ _METADATA_VERSION = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=315,
serialized_end=393,
serialized_start=316,
serialized_end=394,
)
_sym_db.RegisterEnumDescriptor(_METADATA_VERSION)
_METADATA_LANGUAGE = _descriptor.EnumDescriptor(
name='Language',
full_name='legacy_pb.Metadata.Language',
full_name='Metadata.Language',
filename=None,
file=DESCRIPTOR,
values=[
@@ -809,91 +809,91 @@ _METADATA_LANGUAGE = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=396,
serialized_end=1957,
serialized_start=397,
serialized_end=1958,
)
_sym_db.RegisterEnumDescriptor(_METADATA_LANGUAGE)
_METADATA = _descriptor.Descriptor(
name='Metadata',
full_name='legacy_pb.Metadata',
full_name='Metadata',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='version', full_name='legacy_pb.Metadata.version', index=0,
name='version', full_name='Metadata.version', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='language', full_name='legacy_pb.Metadata.language', index=1,
name='language', full_name='Metadata.language', index=1,
number=2, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='title', full_name='legacy_pb.Metadata.title', index=2,
name='title', full_name='Metadata.title', index=2,
number=3, type=9, cpp_type=9, label=2,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='description', full_name='legacy_pb.Metadata.description', index=3,
name='description', full_name='Metadata.description', index=3,
number=4, type=9, cpp_type=9, label=2,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='author', full_name='legacy_pb.Metadata.author', index=4,
name='author', full_name='Metadata.author', index=4,
number=5, type=9, cpp_type=9, label=2,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='license', full_name='legacy_pb.Metadata.license', index=5,
name='license', full_name='Metadata.license', index=5,
number=6, type=9, cpp_type=9, label=2,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='nsfw', full_name='legacy_pb.Metadata.nsfw', index=6,
name='nsfw', full_name='Metadata.nsfw', index=6,
number=7, type=8, cpp_type=7, label=2,
has_default_value=False, default_value=False,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='fee', full_name='legacy_pb.Metadata.fee', index=7,
name='fee', full_name='Metadata.fee', index=7,
number=8, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='thumbnail', full_name='legacy_pb.Metadata.thumbnail', index=8,
name='thumbnail', full_name='Metadata.thumbnail', index=8,
number=9, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='preview', full_name='legacy_pb.Metadata.preview', index=9,
name='preview', full_name='Metadata.preview', index=9,
number=10, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='licenseUrl', full_name='legacy_pb.Metadata.licenseUrl', index=10,
name='licenseUrl', full_name='Metadata.licenseUrl', index=10,
number=11, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
@@ -913,13 +913,13 @@ _METADATA = _descriptor.Descriptor(
extension_ranges=[],
oneofs=[
],
serialized_start=41,
serialized_end=1957,
serialized_start=72,
serialized_end=1958,
)
_METADATA.fields_by_name['version'].enum_type = _METADATA_VERSION
_METADATA.fields_by_name['language'].enum_type = _METADATA_LANGUAGE
_METADATA.fields_by_name['fee'].message_type = fee__pb2._FEE
_METADATA.fields_by_name['fee'].message_type = lbrynet_dot_schema_dot_proto_dot_fee__pb2._FEE
_METADATA_VERSION.containing_type = _METADATA
_METADATA_LANGUAGE.containing_type = _METADATA
DESCRIPTOR.message_types_by_name['Metadata'] = _METADATA
@@ -927,8 +927,8 @@ _sym_db.RegisterFileDescriptor(DESCRIPTOR)
Metadata = _reflection.GeneratedProtocolMessageType('Metadata', (_message.Message,), dict(
DESCRIPTOR = _METADATA,
__module__ = 'metadata_pb2'
# @@protoc_insertion_point(class_scope:legacy_pb.Metadata)
__module__ = 'lbrynet.schema.proto.metadata_pb2'
# @@protoc_insertion_point(class_scope:Metadata)
))
_sym_db.RegisterMessage(Metadata)
+14
View File
@@ -0,0 +1,14 @@
syntax = "proto2";
import "lbrynet/schema/proto/certificate.proto";
message Signature {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required KeyType signatureType = 2;
required bytes signature = 3;
required bytes certificateId = 4;
}
@@ -1,5 +1,5 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: signature.proto
# source: lbrynet/schema/proto/signature.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
@@ -12,23 +12,23 @@ from google.protobuf import symbol_database as _symbol_database
_sym_db = _symbol_database.Default()
from . import certificate_pb2 as certificate__pb2
from lbrynet.schema.proto import certificate_pb2 as lbrynet_dot_schema_dot_proto_dot_certificate__pb2
DESCRIPTOR = _descriptor.FileDescriptor(
name='signature.proto',
package='legacy_pb',
name='lbrynet/schema/proto/signature.proto',
package='',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x0fsignature.proto\x12\tlegacy_pb\x1a\x11\x63\x65rtificate.proto\"\xbb\x01\n\tSignature\x12-\n\x07version\x18\x01 \x02(\x0e\x32\x1c.legacy_pb.Signature.Version\x12)\n\rsignatureType\x18\x02 \x02(\x0e\x32\x12.legacy_pb.KeyType\x12\x11\n\tsignature\x18\x03 \x02(\x0c\x12\x15\n\rcertificateId\x18\x04 \x02(\x0c\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01')
serialized_pb=_b('\n$lbrynet/schema/proto/signature.proto\x1a&lbrynet/schema/proto/certificate.proto\"\xa7\x01\n\tSignature\x12#\n\x07version\x18\x01 \x02(\x0e\x32\x12.Signature.Version\x12\x1f\n\rsignatureType\x18\x02 \x02(\x0e\x32\x08.KeyType\x12\x11\n\tsignature\x18\x03 \x02(\x0c\x12\x15\n\rcertificateId\x18\x04 \x02(\x0c\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01')
,
dependencies=[certificate__pb2.DESCRIPTOR,])
dependencies=[lbrynet_dot_schema_dot_proto_dot_certificate__pb2.DESCRIPTOR,])
_SIGNATURE_VERSION = _descriptor.EnumDescriptor(
name='Version',
full_name='legacy_pb.Signature.Version',
full_name='Signature.Version',
filename=None,
file=DESCRIPTOR,
values=[
@@ -43,42 +43,42 @@ _SIGNATURE_VERSION = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=195,
serialized_end=237,
serialized_start=206,
serialized_end=248,
)
_sym_db.RegisterEnumDescriptor(_SIGNATURE_VERSION)
_SIGNATURE = _descriptor.Descriptor(
name='Signature',
full_name='legacy_pb.Signature',
full_name='Signature',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='version', full_name='legacy_pb.Signature.version', index=0,
name='version', full_name='Signature.version', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='signatureType', full_name='legacy_pb.Signature.signatureType', index=1,
name='signatureType', full_name='Signature.signatureType', index=1,
number=2, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='signature', full_name='legacy_pb.Signature.signature', index=2,
name='signature', full_name='Signature.signature', index=2,
number=3, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='certificateId', full_name='legacy_pb.Signature.certificateId', index=3,
name='certificateId', full_name='Signature.certificateId', index=3,
number=4, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
@@ -97,20 +97,20 @@ _SIGNATURE = _descriptor.Descriptor(
extension_ranges=[],
oneofs=[
],
serialized_start=50,
serialized_end=237,
serialized_start=81,
serialized_end=248,
)
_SIGNATURE.fields_by_name['version'].enum_type = _SIGNATURE_VERSION
_SIGNATURE.fields_by_name['signatureType'].enum_type = certificate__pb2._KEYTYPE
_SIGNATURE.fields_by_name['signatureType'].enum_type = lbrynet_dot_schema_dot_proto_dot_certificate__pb2._KEYTYPE
_SIGNATURE_VERSION.containing_type = _SIGNATURE
DESCRIPTOR.message_types_by_name['Signature'] = _SIGNATURE
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
Signature = _reflection.GeneratedProtocolMessageType('Signature', (_message.Message,), dict(
DESCRIPTOR = _SIGNATURE,
__module__ = 'signature_pb2'
# @@protoc_insertion_point(class_scope:legacy_pb.Signature)
__module__ = 'lbrynet.schema.proto.signature_pb2'
# @@protoc_insertion_point(class_scope:Signature)
))
_sym_db.RegisterMessage(Signature)
+16
View File
@@ -0,0 +1,16 @@
syntax = "proto2";
message Source {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
enum SourceTypes {
UNKNOWN_SOURCE_TYPE = 0;
lbry_sd_hash = 1;
}
required SourceTypes sourceType = 2;
required bytes source = 3;
required string contentType = 4;
}
@@ -1,5 +1,5 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: source.proto
# source: lbrynet/schema/proto/source.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
@@ -15,18 +15,18 @@ _sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='source.proto',
package='legacy_pb',
name='lbrynet/schema/proto/source.proto',
package='',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x0csource.proto\x12\tlegacy_pb\"\xf2\x01\n\x06Source\x12*\n\x07version\x18\x01 \x02(\x0e\x32\x19.legacy_pb.Source.Version\x12\x31\n\nsourceType\x18\x02 \x02(\x0e\x32\x1d.legacy_pb.Source.SourceTypes\x12\x0e\n\x06source\x18\x03 \x02(\x0c\x12\x13\n\x0b\x63ontentType\x18\x04 \x02(\t\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\"8\n\x0bSourceTypes\x12\x17\n\x13UNKNOWN_SOURCE_TYPE\x10\x00\x12\x10\n\x0clbry_sd_hash\x10\x01')
serialized_pb=_b('\n!lbrynet/schema/proto/source.proto\"\xde\x01\n\x06Source\x12 \n\x07version\x18\x01 \x02(\x0e\x32\x0f.Source.Version\x12\'\n\nsourceType\x18\x02 \x02(\x0e\x32\x13.Source.SourceTypes\x12\x0e\n\x06source\x18\x03 \x02(\x0c\x12\x13\n\x0b\x63ontentType\x18\x04 \x02(\t\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01\"8\n\x0bSourceTypes\x12\x17\n\x13UNKNOWN_SOURCE_TYPE\x10\x00\x12\x10\n\x0clbry_sd_hash\x10\x01')
)
_SOURCE_VERSION = _descriptor.EnumDescriptor(
name='Version',
full_name='legacy_pb.Source.Version',
full_name='Source.Version',
filename=None,
file=DESCRIPTOR,
values=[
@@ -41,14 +41,14 @@ _SOURCE_VERSION = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=170,
serialized_end=212,
serialized_start=160,
serialized_end=202,
)
_sym_db.RegisterEnumDescriptor(_SOURCE_VERSION)
_SOURCE_SOURCETYPES = _descriptor.EnumDescriptor(
name='SourceTypes',
full_name='legacy_pb.Source.SourceTypes',
full_name='Source.SourceTypes',
filename=None,
file=DESCRIPTOR,
values=[
@@ -63,42 +63,42 @@ _SOURCE_SOURCETYPES = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=214,
serialized_end=270,
serialized_start=204,
serialized_end=260,
)
_sym_db.RegisterEnumDescriptor(_SOURCE_SOURCETYPES)
_SOURCE = _descriptor.Descriptor(
name='Source',
full_name='legacy_pb.Source',
full_name='Source',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='version', full_name='legacy_pb.Source.version', index=0,
name='version', full_name='Source.version', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='sourceType', full_name='legacy_pb.Source.sourceType', index=1,
name='sourceType', full_name='Source.sourceType', index=1,
number=2, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='source', full_name='legacy_pb.Source.source', index=2,
name='source', full_name='Source.source', index=2,
number=3, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='contentType', full_name='legacy_pb.Source.contentType', index=3,
name='contentType', full_name='Source.contentType', index=3,
number=4, type=9, cpp_type=9, label=2,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
@@ -118,8 +118,8 @@ _SOURCE = _descriptor.Descriptor(
extension_ranges=[],
oneofs=[
],
serialized_start=28,
serialized_end=270,
serialized_start=38,
serialized_end=260,
)
_SOURCE.fields_by_name['version'].enum_type = _SOURCE_VERSION
@@ -131,8 +131,8 @@ _sym_db.RegisterFileDescriptor(DESCRIPTOR)
Source = _reflection.GeneratedProtocolMessageType('Source', (_message.Message,), dict(
DESCRIPTOR = _SOURCE,
__module__ = 'source_pb2'
# @@protoc_insertion_point(class_scope:legacy_pb.Source)
__module__ = 'lbrynet.schema.proto.source_pb2'
# @@protoc_insertion_point(class_scope:Source)
))
_sym_db.RegisterMessage(Source)
+15
View File
@@ -0,0 +1,15 @@
syntax = "proto2";
import "lbrynet/schema/proto/metadata.proto";
import "lbrynet/schema/proto/source.proto";
message Stream {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required Metadata metadata = 2;
required Source source = 3;
}
@@ -1,5 +1,5 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: stream.proto
# source: lbrynet/schema/proto/stream.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
@@ -12,24 +12,24 @@ from google.protobuf import symbol_database as _symbol_database
_sym_db = _symbol_database.Default()
from . import metadata_pb2 as metadata__pb2
from . import source_pb2 as source__pb2
from lbrynet.schema.proto import metadata_pb2 as lbrynet_dot_schema_dot_proto_dot_metadata__pb2
from lbrynet.schema.proto import source_pb2 as lbrynet_dot_schema_dot_proto_dot_source__pb2
DESCRIPTOR = _descriptor.FileDescriptor(
name='stream.proto',
package='legacy_pb',
name='lbrynet/schema/proto/stream.proto',
package='',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x0cstream.proto\x12\tlegacy_pb\x1a\x0emetadata.proto\x1a\x0csource.proto\"\xaa\x01\n\x06Stream\x12*\n\x07version\x18\x01 \x02(\x0e\x32\x19.legacy_pb.Stream.Version\x12%\n\x08metadata\x18\x02 \x02(\x0b\x32\x13.legacy_pb.Metadata\x12!\n\x06source\x18\x03 \x02(\x0b\x32\x11.legacy_pb.Source\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01')
serialized_pb=_b('\n!lbrynet/schema/proto/stream.proto\x1a#lbrynet/schema/proto/metadata.proto\x1a!lbrynet/schema/proto/source.proto\"\x8c\x01\n\x06Stream\x12 \n\x07version\x18\x01 \x02(\x0e\x32\x0f.Stream.Version\x12\x1b\n\x08metadata\x18\x02 \x02(\x0b\x32\t.Metadata\x12\x17\n\x06source\x18\x03 \x02(\x0b\x32\x07.Source\"*\n\x07Version\x12\x13\n\x0fUNKNOWN_VERSION\x10\x00\x12\n\n\x06_0_0_1\x10\x01')
,
dependencies=[metadata__pb2.DESCRIPTOR,source__pb2.DESCRIPTOR,])
dependencies=[lbrynet_dot_schema_dot_proto_dot_metadata__pb2.DESCRIPTOR,lbrynet_dot_schema_dot_proto_dot_source__pb2.DESCRIPTOR,])
_STREAM_VERSION = _descriptor.EnumDescriptor(
name='Version',
full_name='legacy_pb.Stream.Version',
full_name='Stream.Version',
filename=None,
file=DESCRIPTOR,
values=[
@@ -44,35 +44,35 @@ _STREAM_VERSION = _descriptor.EnumDescriptor(
],
containing_type=None,
serialized_options=None,
serialized_start=186,
serialized_end=228,
serialized_start=208,
serialized_end=250,
)
_sym_db.RegisterEnumDescriptor(_STREAM_VERSION)
_STREAM = _descriptor.Descriptor(
name='Stream',
full_name='legacy_pb.Stream',
full_name='Stream',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='version', full_name='legacy_pb.Stream.version', index=0,
name='version', full_name='Stream.version', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='metadata', full_name='legacy_pb.Stream.metadata', index=1,
name='metadata', full_name='Stream.metadata', index=1,
number=2, type=11, cpp_type=10, label=2,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='source', full_name='legacy_pb.Stream.source', index=2,
name='source', full_name='Stream.source', index=2,
number=3, type=11, cpp_type=10, label=2,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
@@ -91,21 +91,21 @@ _STREAM = _descriptor.Descriptor(
extension_ranges=[],
oneofs=[
],
serialized_start=58,
serialized_end=228,
serialized_start=110,
serialized_end=250,
)
_STREAM.fields_by_name['version'].enum_type = _STREAM_VERSION
_STREAM.fields_by_name['metadata'].message_type = metadata__pb2._METADATA
_STREAM.fields_by_name['source'].message_type = source__pb2._SOURCE
_STREAM.fields_by_name['metadata'].message_type = lbrynet_dot_schema_dot_proto_dot_metadata__pb2._METADATA
_STREAM.fields_by_name['source'].message_type = lbrynet_dot_schema_dot_proto_dot_source__pb2._SOURCE
_STREAM_VERSION.containing_type = _STREAM
DESCRIPTOR.message_types_by_name['Stream'] = _STREAM
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
Stream = _reflection.GeneratedProtocolMessageType('Stream', (_message.Message,), dict(
DESCRIPTOR = _STREAM,
__module__ = 'stream_pb2'
# @@protoc_insertion_point(class_scope:legacy_pb.Stream)
__module__ = 'lbrynet.schema.proto.stream_pb2'
# @@protoc_insertion_point(class_scope:Stream)
))
_sym_db.RegisterMessage(Stream)
+102
View File
@@ -0,0 +1,102 @@
V_0_0_1 = "_0_0_1"
V_0_0_2 = "_0_0_2"
V_0_0_3 = "_0_0_3"
V_0_1_0 = "_0_1_0"
VERSION_MAP = {
V_0_0_1: 1,
V_0_0_2: 2,
V_0_0_3: 3,
V_0_1_0: 4,
}
VERSION_NAMES = {
1: V_0_0_1,
2: V_0_0_2,
3: V_0_0_3,
4: V_0_1_0
}
LBC = "LBC"
BTC = "BTC"
USD = "USD"
CURRENCY_MAP = {
LBC: 1,
BTC: 2,
USD: 3
}
CURRENCY_NAMES = {
1: LBC,
2: BTC,
3: USD
}
ADDRESS_LENGTH = 25
ADDRESS_CHECKSUM_LENGTH = 4
VERSION = "version"
STREAM_TYPE = "streamType"
CERTIFICATE_TYPE = "certificateType"
CLAIM_TYPE = "claimType"
SIGNATURE = "publisherSignature"
CLAIM_TYPES = {
STREAM_TYPE: "stream",
CERTIFICATE_TYPE: "certificate"
}
CLAIM_TYPE_NAMES = {
1: "stream",
2: "certificate"
}
LBRY_SD_HASH = "lbry_sd_hash"
LBRY_SD_HASH_LENGTH = 48
SOURCE_TYPES = {
LBRY_SD_HASH: 1
}
NIST256p = "NIST256p"
NIST384p = "NIST384p"
SECP256k1 = "SECP256k1"
ECDSA_CURVES = {
NIST256p: 1,
NIST384p: 2,
SECP256k1: 3
}
CURVE_NAMES = {
1: NIST256p,
2: NIST384p,
3: SECP256k1
}
SHA256 = "sha256"
SHA384 = "sha384"
B58_CHARS = '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'
assert len(B58_CHARS) == 58
PUBKEY_ADDRESS = 0
SCRIPT_ADDRESS = 5
ADDRESS_PREFIXES = {
"lbrycrd_main": {
PUBKEY_ADDRESS: 85,
SCRIPT_ADDRESS: 122
},
"lbrycrd_regtest": {
PUBKEY_ADDRESS: 111,
SCRIPT_ADDRESS: 196
},
"lbrycrd_testnet": {
PUBKEY_ADDRESS: 111,
SCRIPT_ADDRESS: 196
},
}
+54
View File
@@ -0,0 +1,54 @@
from copy import deepcopy
from lbrynet.schema.proto import certificate_pb2 as cert_pb
from lbrynet.schema.schema.schema import Schema
from lbrynet.schema.schema import VERSION_MAP, V_0_0_1, ECDSA_CURVES, CURVE_NAMES
from lbrynet.schema.validator import get_key_type_from_dem
class _ECDSAKeyHelper(object):
def __init__(self, key):
self._key = key
@property
def der(self):
return self._key.to_der()
@property
def curve_name(self):
return self._key.curve.name
class Certificate(Schema):
@classmethod
def load(cls, message):
_key = deepcopy(message)
_message_pb = cert_pb.Certificate()
if isinstance(_key, dict):
_message_pb.publicKey = _key.pop("publicKey")
_message_pb.version = VERSION_MAP[_key.pop("version")]
_message_pb.keyType = ECDSA_CURVES[_key.pop("keyType")]
else:
_message_pb.version = _key.version
_message_pb.keyType = _key.keyType
_message_pb.publicKey = _key.publicKey
if _message_pb.keyType not in CURVE_NAMES:
raise Exception("Unknown curve")
if get_key_type_from_dem(_message_pb.publicKey) != _message_pb.keyType:
raise Exception("Curve mismatch")
return cls._load(_key, _message_pb)
@classmethod
def load_from_key_obj(cls, key, key_type):
if key_type in ECDSA_CURVES:
_key = _ECDSAKeyHelper(key)
else:
raise Exception("Unknown key type: %s" % str(type(key)))
if key_type != _key.curve_name:
raise Exception("Curve mismatch")
msg = {
"version": V_0_0_1,
"keyType": key_type,
"publicKey": _key.der,
}
return cls.load(msg)
+51
View File
@@ -0,0 +1,51 @@
from copy import deepcopy
from lbrynet.schema.proto import claim_pb2 as claim_pb
from lbrynet.schema.schema import VERSION_MAP
from lbrynet.schema.schema.signature import Signature
from lbrynet.schema.schema.certificate import Certificate
from lbrynet.schema.schema.schema import Schema
from lbrynet.schema.schema.stream import Stream
class Claim(Schema):
CLAIM_TYPE_STREAM = 1
CLAIM_TYPE_CERT = 2
@classmethod
def load(cls, message):
_claim = deepcopy(message)
_message_pb = claim_pb.Claim()
_message_pb.version = VERSION_MAP[_claim.pop("version")]
if "certificate" in _claim:
_cert = _claim.pop("certificate")
if isinstance(_cert, dict):
cert = Certificate.load(_cert)
else:
cert = _cert
claim_type = Claim.CLAIM_TYPE_CERT
_message_pb.certificate.MergeFrom(cert)
elif "stream" in _claim:
_stream = _claim.pop("stream")
if isinstance(_stream, dict):
stream = Stream.load(_stream)
else:
stream = _stream
claim_type = Claim.CLAIM_TYPE_STREAM
_message_pb.stream.MergeFrom(stream)
else:
raise AttributeError
_message_pb.claimType = claim_type
if "publisherSignature" in _claim:
_publisherSignature = _claim.pop("publisherSignature")
if isinstance(_publisherSignature, dict):
publisherSignature = Signature.load(_publisherSignature)
else:
publisherSignature = _publisherSignature
_message_pb.publisherSignature.MergeFrom(publisherSignature)
return cls._load(_claim, _message_pb)
+17
View File
@@ -0,0 +1,17 @@
from copy import deepcopy
from lbrynet.schema.schema.schema import Schema
from lbrynet.schema.proto import fee_pb2 as fee_pb
from lbrynet.schema.schema import VERSION_MAP, CURRENCY_MAP
class Fee(Schema):
@classmethod
def load(cls, message):
_fee = deepcopy(message)
currency = CURRENCY_MAP[_fee.pop('currency')]
_message_pb = fee_pb.Fee()
_message_pb.version = VERSION_MAP[_fee.pop("version")]
_message_pb.currency = currency
_message_pb.address = _fee.pop('address')
return cls._load(_fee, _message_pb)
+17
View File
@@ -0,0 +1,17 @@
from copy import deepcopy
from lbrynet.schema.proto import metadata_pb2 as metadata_pb
from lbrynet.schema.schema.fee import Fee
from lbrynet.schema.schema.schema import Schema
from lbrynet.schema.schema import VERSION_MAP
class Metadata(Schema):
@classmethod
def load(cls, message):
_metadata = deepcopy(message)
_message_pb = metadata_pb.Metadata()
_message_pb.version = VERSION_MAP[_metadata.pop("version")]
if 'fee' in _metadata:
fee_pb = Fee.load(_metadata.pop('fee'))
_message_pb.fee.CopyFrom(fee_pb)
return cls._load(_metadata, _message_pb)
+15
View File
@@ -0,0 +1,15 @@
import json
import google.protobuf.json_format as json_pb # pylint: disable=no-name-in-module
from google.protobuf.message import Message # pylint: disable=no-name-in-module,import-error
class Schema(Message):
@classmethod
def load(cls, message):
raise NotImplementedError
@classmethod
def _load(cls, data, message):
if isinstance(data, dict):
data = json.dumps(data)
return json_pb.Parse(data, message)
+17
View File
@@ -0,0 +1,17 @@
from copy import deepcopy
from lbrynet.schema.proto import signature_pb2 as signature_pb
from lbrynet.schema.schema import VERSION_MAP, ECDSA_CURVES
from lbrynet.schema.schema.schema import Schema
class Signature(Schema):
@classmethod
def load(cls, message):
_signature = deepcopy(message)
_message_pb = signature_pb.Signature()
_message_pb.version = VERSION_MAP[_signature.pop("version")]
_message_pb.signatureType = ECDSA_CURVES[_signature.pop("signatureType")]
_message_pb.certificateId = _signature.pop("certificateId")
_message_pb.signature = _signature.pop("signature")
return cls._load(_signature, _message_pb)
+19
View File
@@ -0,0 +1,19 @@
from copy import deepcopy
from lbrynet.schema.proto import source_pb2 as source_pb
from lbrynet.schema.schema import SOURCE_TYPES, LBRY_SD_HASH_LENGTH, VERSION_MAP
from lbrynet.schema.schema.schema import Schema
from lbrynet.schema.error import InvalidSourceHashLength
class Source(Schema):
@classmethod
def load(cls, message):
_source = deepcopy(message)
sd_hash = _source.pop('source')
assert len(sd_hash) == LBRY_SD_HASH_LENGTH, InvalidSourceHashLength(len(sd_hash))
_message_pb = source_pb.Source()
_message_pb.version = VERSION_MAP[_source.pop("version")]
_message_pb.sourceType = SOURCE_TYPES[_source.pop('sourceType')]
_message_pb.source = sd_hash
_message_pb.contentType = _source.pop('contentType')
return cls._load(_source, _message_pb)
+20
View File
@@ -0,0 +1,20 @@
from copy import deepcopy
from lbrynet.schema.schema.source import Source
from lbrynet.schema.proto import stream_pb2 as stream_pb
from lbrynet.schema.schema import VERSION_MAP
from lbrynet.schema.schema.metadata import Metadata
from lbrynet.schema.schema.schema import Schema
class Stream(Schema):
@classmethod
def load(cls, message):
_claim = deepcopy(message)
source = Source.load(_claim.pop('source'))
metadata = Metadata.load(_claim.pop('metadata'))
_message_pb = stream_pb.Stream()
_message_pb.version = VERSION_MAP[_claim.pop("version")]
_message_pb.source.CopyFrom(source)
_message_pb.metadata.CopyFrom(metadata)
return cls._load(_claim, _message_pb)
+44
View File
@@ -0,0 +1,44 @@
from collections import namedtuple
from typing import Union
LEGACY = namedtuple('Legacy', 'payload')
NAMED_SECP256K1 = namedtuple('NamedSECP256k1', 'raw_signature certificate_id payload')
FLAGS = {
LEGACY: 0x80,
NAMED_SECP256K1: 0x01
}
class Signature:
def __init__(self, data: Union[LEGACY, NAMED_SECP256K1]):
self.data = data
@property
def payload(self):
return self.data.payload
@property
def certificate_id(self):
if type(self.data) == NAMED_SECP256K1:
return self.data.certificate_id
@property
def raw_signature(self):
if type(self.data) == NAMED_SECP256K1:
return self.data.raw_signature
@classmethod
def flagged_parse(cls, binary: bytes):
flag = binary[0]
if flag == FLAGS[NAMED_SECP256K1]:
return cls(NAMED_SECP256K1(binary[1:65], binary[65:85], binary[85:]))
else:
return cls(LEGACY(binary))
@property
def serialized(self):
if isinstance(self.data, NAMED_SECP256K1):
return (bytes([FLAGS[type(self.data)]]) + self.data.raw_signature + self.data.certificate_id + self.payload)
elif isinstance(self.data, LEGACY):
return self.payload
Loaded 100 of 169 files, more files were not shown because too many files have changed in this diff. Show more