aljaz/pear-docs
1
0
Fork 0
mirror of https://github.com/aljazceru/pear-docs.git synced 2025-12-17 14:34:19 +01:00
Code Issues Packages Projects Releases Wiki Activity

pearify

Browse Source
This commit is contained in:
dmc
2024-01-31 23:43:56 +01:00
parent 3b4264dea3
commit 95145e31cd
4 changed files with 73 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
howto/connect-two-peers-by-key-with-hyperdht.md
View File

@@ -10,7 +10,7 @@ pear init -y -t terminal
npm install hyperdht b4a graceful-goodbye npm install hyperdht b4a graceful-goodbye
``` ```
[Hyperswarm](../building-blocks/hyperswarm.md) helps to find and connect to peers who are announcing a common 'topic'. The swarm topic can be anything. The HyperDHT uses a series of holepunching techniques to establish direct connections between peers, even if they're located on home networks with tricky NATs. [`Hyperswarm`](../building-blocks/hyperswarm.md) helps to find and connect to peers who are announcing a common 'topic'. The swarm topic can be anything. The HyperDHT uses a series of holepunching techniques to establish direct connections between peers, even if they're located on home networks with tricky NATs.
In the HyperDHT, peers are identified by a public key, not by an IP address. With the public key, users can connect to each other irrespective of their location, even if they move between different networks. In the HyperDHT, peers are identified by a public key, not by an IP address. With the public key, users can connect to each other irrespective of their location, even if they move between different networks.

28
howto/replicate-and-persist-with-hypercore.md
View File

@@ -2,7 +2,7 @@
In the HyperDHT How-to ([Connect Two Peers](./connect-two-peers-by-key-with-hyperdht.md)) and the Hyperswarm How-to ([Connect Many Peers](./connect-to-many-peers-by-topic-with-hyperswarm.md)), peers can exchange chat messages so long as both are online at the same time and directly connected. The application is ephemeral, the messages are not persisted - they will be lost if the recipient is offline. Hypercore provides the persistence. In the HyperDHT How-to ([Connect Two Peers](./connect-two-peers-by-key-with-hyperdht.md)) and the Hyperswarm How-to ([Connect Many Peers](./connect-to-many-peers-by-topic-with-hyperswarm.md)), peers can exchange chat messages so long as both are online at the same time and directly connected. The application is ephemeral, the messages are not persisted - they will be lost if the recipient is offline. Hypercore provides the persistence.
[Hypercore](../building-blocks/hypercore.md) is a secure, distributed append-only log. It is built for sharing enormous datasets and streams of real-time data. It has a secure transport protocol, making it easy to build fast and scalable peer-to-peer applications. [`Hypercore`](../building-blocks/hypercore.md) is a secure, distributed append-only log. It is built for sharing enormous datasets and streams of real-time data. It has a secure transport protocol, making it easy to build fast and scalable peer-to-peer applications.
Now extend the ephemeral chat example above but using Hypercore to add many significant new features: Now extend the ephemeral chat example above but using Hypercore to add many significant new features:
@@ -20,12 +20,12 @@ Create the `writer-app` project with these commands:
mkdir writer-app mkdir writer-app
cd writer-app cd writer-app
pear init -y -t terminal pear init -y -t terminal
npm install hypercore hyperswarm b4a
``` ```
Alter the generated `writer-app/index.js` to the following: Alter the generated `writer-app/index.js` to the following:
```javascript ```javascript
import path from 'path'
import Hyperswarm from 'hyperswarm' import Hyperswarm from 'hyperswarm'
import Hypercore from 'hypercore' import Hypercore from 'hypercore'
import b4a from 'b4a' import b4a from 'b4a'
@@ -33,7 +33,7 @@ import b4a from 'b4a'
const swarm = new Hyperswarm() const swarm = new Hyperswarm()
Pear.teardown(() => swarm.destroy()) Pear.teardown(() => swarm.destroy())
const core = new Hypercore(path.join(Pear.config.storage, 'writer-storage')) const core = new Hypercore(Pear.config.storage)
// core.key and core.discoveryKey will only be set after core.ready resolves // core.key and core.discoveryKey will only be set after core.ready resolves
await core.ready() await core.ready()
@@ -57,6 +57,7 @@ Create the `reader-app` project with these commands:
mkdir reader-app mkdir reader-app
cd reader-app cd reader-app
pear init -y -t terminal pear init -y -t terminal
npm install hypercore hyperswarm
``` ```
Alter the generated `reader-app/index.js` to the following: Alter the generated `reader-app/index.js` to the following:
@@ -65,12 +66,11 @@ Alter the generated `reader-app/index.js` to the following:
```javascript ```javascript
import Hyperswarm from 'hyperswarm' import Hyperswarm from 'hyperswarm'
import Hypercore from 'hypercore' import Hypercore from 'hypercore'
import goodbye from 'graceful-goodbye'
const swarm = new Hyperswarm() const swarm = new Hyperswarm()
goodbye(() => swarm.destroy()) Pear.teardown(() => swarm.destroy())
const core = new Hypercore('./reader-storage', process.argv[2]) const core = new Hypercore(Pear.config.storage, process.argv[2])
await core.ready() await core.ready()
const foundPeers = core.findingPeers() const foundPeers = core.findingPeers()
@@ -94,22 +94,20 @@ for await (const block of core.createReadStream({ start: core.length, live: true
} }
``` ```
There should now be two folders, `writer-app` and `reader-app`. In one terminal, open `writer-app` with `pear dev`.
In one terminal run:
``` ```
pear dev ./writer-app cd writer-app
pear dev
``` ```
If the `writer-app` is not in the current dir, adjust the path to point to `writer-app`. The `writer-app` will output the Hypercore key.
In another terminal run In another terminal, open the `reader-app` and pass it the key:
``` ```
pear dev ./reader-app cd reader-app
pear dev -- <SUPPLY THE KEY HERE>
``` ```
Again if the `reader-app` is not in the current dir, adjust the path to point to `reader-app`.
As inputs are made to the terminal running the writer application, outputs should be shown in the terminal running the reader application. As inputs are made to the terminal running the writer application, outputs should be shown in the terminal running the reader application.

11
howto/share-append-only-databases-with-hyperbee.md
View File

@@ -1,16 +1,7 @@
# How to share Append-Only Databases with Hyperbee # How to share Append-Only Databases with Hyperbee
Get setup by creating a project folder and installing dependencies: [Hyperbee](../building-blocks/hyperbee.md) is an append-only B-tree based on Hypercore. It provides a key/value-store API with methods to insert and get key/value pairs, perform atomic batch insertions, and create sorted iterators.
```bash
mkdir connect-two-peers
cd connect-two-peers
pear init -y -t terminal
npm install hyperdht b4a graceful-goodbye
```
[hyperbee.md](../building-blocks/hyperbee.md) is an append-only B-tree based on Hypercore. It provides a key/value-store API with methods to insert and get key/value pairs, perform atomic batch insertions, and create sorted iterators.
The example consists of three files: `writer.js` , `bee-reader.js` and `core-reader.js`. The example consists of three files: `writer.js` , `bee-reader.js` and `core-reader.js`.

88
howto/work-with-many-hypercores-using-corestore.md
View File

@@ -1,34 +1,31 @@
# How to work with many Hypercores using Corestore # How to work with many Hypercores using Corestore
Get setup by creating a project folder and installing dependencies:
```bash
mkdir many-cores
cd many-cores
pear init -y -t terminal
npm install corestore hyperswarm b4a graceful-goodbye
```
An append-only log is powerful on its own, but it's most useful as a building block for constructing larger data structures, such as databases or filesystems. Building these data structures often requires many cores, each with different responsibilities. For example, Hyperdrive uses one core to store file metadata and another to store file contents. An append-only log is powerful on its own, but it's most useful as a building block for constructing larger data structures, such as databases or filesystems. Building these data structures often requires many cores, each with different responsibilities. For example, Hyperdrive uses one core to store file metadata and another to store file contents.
[corestore.md](../helpers/corestore.md) is a Hypercore factory that makes it easier to manage large collections of named Hypercores. A simple example below demonstrates a pattern often in use: co-replicating many cores using Corestore, where several 'internal cores' are linked to from a primary core. Only the primary core is announced on the swarm -- the keys for the others are recorded inside of that core. [`Corestore`](../helpers/corestore.md) is a Hypercore factory that makes it easier to manage large collections of named Hypercores. This how-to demonstrates a pattern often in use: co-replicating many cores using Corestore, where several 'internal cores' are linked to from a primary core. Only the primary core is announced on the swarm -- the keys for the others are recorded inside of that core.
This example consists of two files: `writer.js` and `reader.js`. In the previous example, we replicated only a single Hypercore instance. But in this example, we will replicate a single Corestore instance, which will internally manage the replication of a collection of Hypercores. In [How to replicate and persist with Hypercore](./replicate-and-persist-with-hypercore.md), only single Hypercore instance was replicated. But in this how-to, we will replicate a single Corestore instance, which will internally manage the replication of a collection of Hypercores. We will achieve this with two Pear Terminal Applications: `multicore-writer-app` and `multicore-reader-app`.
The file `writer.js` uses a Corestore instance to create three Hypercores, which are then replicated with other peers using Hyperswarm. The keys for the second and third cores are stored in the first core (the first core 'bootstraps' the system). Messages entered into the command line are written into the second and third cores, depending on the length of the message. To execute `reader.js`, copy the main core key logged into the command line. Create the `multicore-writer-app` project with these commands:
```
mkdir multicore-writer-app
cd multicore-writer-app
pear init -y -t terminal
npm install corestore hyperswarm b4a
```
Alter the generated `multicore-writer-app/index.js` to the following
```javascript ```javascript
//writer.js
import Corestore from 'corestore'
import Hyperswarm from 'hyperswarm' import Hyperswarm from 'hyperswarm'
import goodbye from 'graceful-goodbye' import Corestore from 'corestore'
import b4a from 'b4a' import b4a from 'b4a'
const store = new Corestore('./writer-storage') const store = new Corestore(Pear.config.storage)
const swarm = new Hyperswarm() const swarm = new Hyperswarm()
goodbye(() => swarm.destroy()) Pear.teardown(() => swarm.destroy())
// A name is a purely-local, and maps to a key pair. It's not visible to readers. // A name is a purely-local, and maps to a key pair. It's not visible to readers.
// Since a name always corresponds to a key pair, these are all writable // Since a name always corresponds to a key pair, these are all writable
@@ -45,18 +42,18 @@ swarm.join(core1.discoveryKey)
// Corestore replication internally manages to replicate every loaded core // Corestore replication internally manages to replicate every loaded core
// Corestore *does not* exchange keys (read capabilities) during replication. // Corestore *does not* exchange keys (read capabilities) during replication.
swarm.on('connection', conn => store.replicate(conn)) swarm.on('connection', (conn) => store.replicate(conn))
// Since Corestore does not exchange keys, they need to be exchanged elsewhere. // Since Corestore does not exchange keys, they need to be exchanged elsewhere.
// Here, we'll record the other keys in the first block of core1. // Here, we'll record the other keys in the first block of core1.
if (core1.length === 0) { if (core1.length === 0) {
await core1.append({ await core1.append({
otherKeys: [core2, core3].map(core => b4a.toString(core.key, 'hex')) otherKeys: [core2, core3].map((core) => b4a.toString(core.key, 'hex'))
}) })
} }
// Record all short messages in core2, and all long ones in core3 // Record all short messages in core2, and all long ones in core3
process.stdin.on('data', data => { Pear.stdio.in.on('data', (data) => {
if (data.length < 5) { if (data.length < 5) {
console.log('appending short data to core2') console.log('appending short data to core2')
core2.append(data) core2.append(data)
@@ -67,36 +64,46 @@ process.stdin.on('data', data => {
}) })
``` ```
`reader.js` connects to the previous peer with Hyperswarm and replicates the local Corestore instance to receive the data from it. This requires the copied key to be supplied as an argument when executing the file, which will then be used to create a core with the same public key as the other peer (i.e., the same discovery key for both the reader and writer peers). The `multicore-writer-app` uses a Corestore instance to create three Hypercores, which are then replicated with other peers using `Hyperswarm`. The keys for the second and third cores are stored in the first core (the first core bootstraps the system). Messages entered into the command line are written into the second and third cores, depending on the length of the message. The main core key logged into the command line so that it can be passed to the `multicore-reader-app`.
The `multicore-reader-app` connects to the previous peer with `Hyperswarm` and replicates the local `Corestore` instance to receive the data from it. This requires the copied key to be supplied as an argument when executing the file, which will then be used to create a core with the same public key as the other peer (i.e., the same discovery key for both the reader and writer peers).
```
mkdir multicore-reader-app
cd multicore-reader-app
pear init -y -t terminal
npm install corestore hyperswarm b4a
```
Alter the generated `multicore-reader-app/index.js` to the following
```javascript ```javascript
//reader.js
import Corestore from 'corestore' import Corestore from 'corestore'
import Hyperswarm from 'hyperswarm' import Hyperswarm from 'hyperswarm'
import goodbye from 'graceful-goodbye'
import b4a from 'b4a' import b4a from 'b4a'
// pass the key as a command line argument if (!Pear.config.args[0]) throw new Error('provide a key')
const key = b4a.from(process.argv[2], 'hex')
// creation of a Corestore instance const key = b4a.from(Pear.config.args[0], 'hex')
const store = new Corestore('./reader-storage')
const store = new Corestore(Pear.config.storage)
await store.ready()
const swarm = new Hyperswarm() const swarm = new Hyperswarm()
goodbye(() => swarm.destroy()) Pear.teardown(() => swarm.destroy())
// replication of corestore instance on every connection // replication of corestore instance on every connection
swarm.on('connection', conn => store.replicate(conn)) swarm.on('connection', (conn) => store.replicate(conn))
// creation/getting of a hypercore instance using the key passed // creation/getting of a hypercore instance using the key passed
const core = store.get({ key, valueEncoding: 'json' }) const core = store.get({ key, valueEncoding: 'json' })
// wait till all the properties of the hypercore instance are initialized // wait till all the properties of the hypercore instance are initialized
await core.ready() await core.ready()
const foundPeers = store.findingPeers() const foundPeers = core.findingPeers()
// join a topic
swarm.join(core.discoveryKey) swarm.join(core.discoveryKey)
swarm.on('connection', conn => core.replicate(conn))
swarm.flush().then(() => foundPeers()) swarm.flush().then(() => foundPeers())
// update the meta-data of the hypercore instance // update the meta-data of the hypercore instance
@@ -121,3 +128,22 @@ for (const key of otherKeys) {
}) })
} }
``` ```
In one terminal, open `multicore-writer-app` with `pear dev`.
```
cd mutlicore-writer-app
pear dev
```
The `multicore-writer-app` will output the main core key.
In another terminal, open the `multicore-reader-app` and pass it the key:
```
cd reader-app
pear dev -- <SUPPLY THE KEY HERE>
```
As inputs are made to the terminal running the writer application, outputs should be shown in the terminal running the reader application.