diff --git a/docs/storage/README.md b/docs/storage/README.md index e69de29..5cd4cf4 100644 --- a/docs/storage/README.md +++ b/docs/storage/README.md @@ -0,0 +1,22 @@ +# Welcome to Logos Storage + +Logos Storage aims at being a resilient, decentralized and censorship-resistance storage layer for +the Logos stack. At present, Logos Storage provides a basic filesharing layer, not unlike IPFS or +Bittorrent, which allows users to publish and share files with one another. + +Moving forward, Logos Storage will enable full provider and downloader anonymity, as well as +anonymous persistence. Check out [our roadmap](https://roadmap.logos.co/storage/roadmap/) for more details. + +## Using Logos Storage + +Logos Storage can currently be used in the following ways: + +**Logos Storage UI App.** + +* The [Logos Storage UI App](https://github.com/logos-co/logos-storage-ui) is a desktop application which allows one to upload and download files over the Logos Storage network. This is the easiest, simplest way to try out Logos Storage. + +**Programmatically.** + +* **Logos Module API.** The [Logos Storage Module](https://github.com/logos-co/logos-storage-module) - a high-level C++ API - is the recommended way of using Logos Storage in your app. Make sure to check: [[API Tutorial](https://logos-co.github.io/logos-storage-module/api_tutorial.html) | [API reference](https://logos-co.github.io/logos-storage-module/api_reference.html)] + +* **libstorage.** [libstorage](https://github.com/logos-storage/logos-storage-nim) is a lower-level C API that goes under the Logos Module API. If you need to construct bindings for a language other than C++, this is what you would use. We have [Go](https://github.com/logos-storage/logos-storage-go), [Rust](https://github.com/nipsysdev/storage-rust-bindings), and higher-level, simple [C](https://github.com/logos-storage/easylibstorage) bindings available. [[API tutorial](./libstorage-tutorial.md) | [API reference](https://github.com/logos-storage/logos-storage-nim/blob/master/library/README.md)] diff --git a/docs/storage/libstorage-tutorial.md b/docs/storage/libstorage-tutorial.md new file mode 100644 index 0000000..0472d63 --- /dev/null +++ b/docs/storage/libstorage-tutorial.md @@ -0,0 +1,210 @@ +--- +title: Build a CLI app with Logos Storage +doc_type: procedure +product: storage +topics: [] +steps_layout: sectioned +authors: +owner: logos +doc_version: 1 +slug: libstorage-tutorial +--- + +# Build a CLI app with Logos Storage + +#### Get started building a CLI application that transfers files over the Logos Storage network. + +This tutorial walks you through building a simple CLI application that uploads and downloads files over the Logos Storage network using the [Logos Storage Module API v0.3.2](https://logos-co.github.io/logos-storage-module/api_reference.html). It is intended for developers who are setting up a new application using the skeleton project and working through the module lifecycle for the first time. + +The tutorial uses the [Logos Storage App Skeleton](https://github.com/logos-storage/logos-storage-app-skeleton), which provides a ready-made entry point at `app_main` that uses the `LogosModules` object to access the API. The skeleton also provides a set of Qt-compatible synchronization utilities. + +**Before you start**, make sure you have the following: + +- Nix package manager +- Git + +## What to expect + +- You can initialize, start, and cleanly shut down the storage module within your application. +- You can upload a file to the network and receive a Content Identifier (CID) that uniquely identifies it. +- You can download a file from the network using a CID and a Signed Peer Record (SPR). + +## Step 1: Build the skeleton app + +Clone the skeleton repository and compile the binary so you have a working entry point before adding any storage logic. + +1. Clone the skeleton repository: + + ```bash + git clone https://github.com/logos-storage/logos-storage-app-skeleton.git + cd logos-storage-app-skeleton + ``` + +1. Build with Nix: + + ```bash + nix build + ``` + +1. Confirm the compiled binary is available at `./result/bin/storage-app`. + +## Step 2: Initialize the module + +Call `init()` with your configuration once at startup, passing a JSON configuration string. See the [API Reference]((https://logos-co.github.io/logos-storage-module/api_reference.html)) for all available options. + + ```cpp + const QString jsonConfig = "{}"; + bool result = m_logos->storage_module.init(jsonConfig); + ``` + +> **Caution** +> +> Do not call `init()` more than once per instance unless you call `destroy()` first. + +## Step 3: Start the node + +Subscribe to the `storageStart` event and call `start()` afterward, to be able to detect failures. + + ```cpp + m_logos->storage_module.on("storageStart", [this](const QVariantList& data) { + bool success = data[0].toBool(); + if (!success) { + QString error = data[1].toString(); + // Handle error + } + }); + + bool result = m_logos->storage_module.start(); + ``` + +## Step 4: Upload a file + +The Storage Module allows for two upload approaches. Choose `uploadUrl` for straightforward cases. Use the streaming API when you need fine-grained control over how data is sent. + +### Upload with `uploadUrl` + +The simplest way to upload files is to subscribe to the upload events, then call `uploadUrl()` with the path to your file. The network returns a Content Identifier (CID) on success. + +1. Subscribe to the `storageUploadDone` and `storageUploadProgress` events: + + ```cpp + //m_logos is the LogosModules object, used for API calls + m_logos->storage_module.on("storageUploadDone", [this](const QVariantList& data) { + bool success = data[0].toBool(); + QString sessionId = data[1].toString(); + QString cidOrError = data[2].toString(); + + if (success) { + qDebug() << "Upload complete. CID:" << cidOrError; + } else { + qDebug() << "Upload failed:" << cidOrError; + } + }); + + m_logos->storage_module.on("storageUploadProgress", [this](const QVariantList& data) { + bool success = data[0].toBool(); + QString sessionId = data[1].toString(); + int bytes = data[2].toInt(); + qDebug() << "Uploaded" << bytes << "bytes"; + }); + ``` + +1. Call `uploadUrl()` with the local file path: + + ```cpp + QUrl fileUrl = QUrl::fromLocalFile("/path/to/myfile"); + LogosResult result = m_logos->storage_module.uploadUrl(fileUrl); + ``` + +### Upload with the streaming API + +Use the streaming upload API for chunk-level control. + +1. Initialize the session: + ```cpp + LogosResult result = m_logos->storage_module.uploadInit(filename); + QString sessionId = result.getValue(); + ``` + +1. . Upload chunks: + ```cpp + QFile file(filepath); + file.open(QIODevice::ReadOnly); + int chunkSize = 1024 * 64; + while (!file.atEnd()) { + QByteArray chunk = file.read(chunkSize); + result = m_logos->storage_module.uploadChunk(sessionId, chunk); + if (!result.success) { + // Handle error + break; + } + } + ``` + +1. Finalize + ```cpp + result = m_logos->storage_module.uploadFinalize(sessionId); + if (result.success) { + QString cid = result.getValue(); + qDebug() << "CID:" << cid; + } + ``` + +## Step 5: Download a file + +To download content, you need the CID returned during upload and the Signed Peer Record (SPR) of a node that holds the content. + +1. Subscribe to the `storageDownloadDone` and `storageDownloadProgress` events: + + ```cpp + //m_logos is the LogosModules object, used for API calls + m_logos->storage_module.on("storageDownloadDone", [this](const QVariantList& data) { + bool success = data[0].toBool(); + QString message = data[1].toString(); + if (success) { + qDebug() << "Download complete"; + } else { + qDebug() << "Download failed:" << message; + } + }); + + m_logos->storage_module.on("storageDownloadProgress", [this](const QVariantList& data) { + bool success = data[0].toBool(); + QString sessionId = data[1].toString(); + int size = data[2].toInt(); + qDebug() << "Downloaded" << size << "bytes"; + }); + ``` + +1. Call `downloadToUrl()` with the CID and the local destination path: + + ```cpp + QUrl destination = QUrl::fromLocalFile("/path/to/output"); + LogosResult result = m_logos->storage_module.downloadToUrl(cid, destination /*, local = false*/); + ``` + + - Set the `local` (third) parameter of `downloadToUrl` to `true` to retrieve only locally-cached data. + - Leave `local` as `false` (default) to fetch from the network. + +## Step 6: Stop and clean up + +Always stop the node before destroying resources to avoid leaving sessions open. To do so, call `stop()` and wait for the `storageStop` event, then call `destroy()`: + + ```cpp + LogosResult result = m_logos->storage_module.stop(); + // Wait for storageStop event... + result = m_logos->storage_module.destroy(); + ``` + +## Frequently asked questions + +### Can I run the storage module without a UI? + +Yes. The storage module supports headless mode, which lets you run it from the command line: + +```bash +./logos/bin/logoscore -m ./modules --load-modules storage_module \ + -c "storage_module.init(@config.json)" \ + -c "storage_module.start()" \ + -c "storage_module.importFiles(/path/to/files)" +``` \ No newline at end of file