- Revolt Backend
+ Stoat Backend
- [](https://github.com/revoltchat/backend/stargazers)
- [](https://github.com/revoltchat/backend/network/members)
- [](https://github.com/revoltchat/backend/pulls)
- [](https://github.com/revoltchat/backend/issues)
- [](https://github.com/revoltchat/backend/graphs/contributors)
- [](https://github.com/revoltchat/backend/blob/main/LICENSE)
+ [](https://github.com/stoatchat/stoatchat/stargazers)
+ [](https://github.com/stoatchat/stoatchat/network/members)
+ [](https://github.com/stoatchat/stoatchat/pulls)
+ [](https://github.com/stoatchat/stoatchat/issues)
+ [](https://github.com/stoatchat/stoatchat/graphs/contributors)
+ [](https://github.com/stoatchat/stoatchat/blob/main/LICENSE)
-The services and libraries that power the Revolt service.
+The services and libraries that power the Stoat service.
| Crate | Path | Description | |
@@ -21,9 +21,11 @@ The services and libraries that power the Revolt service.
| `core/permissions` | [crates/core/permissions](crates/core/permissions) | Core: Permission Logic |     |
| `core/presence` | [crates/core/presence](crates/core/presence) | Core: User Presence |     |
| `core/result` | [crates/core/result](crates/core/result) | Core: Result and Error types |     |
+| `core/coalesced` | [crates/core/coalesced](crates/core/coalesced) | Core: Coalescion service |     |
| `delta` | [crates/delta](crates/delta) | REST API server |  |
| `bonfire` | [crates/bonfire](crates/bonfire) | WebSocket events server |  |
| `services/january` | [crates/services/january](crates/services/january) | Proxy server |  |
+| `services/gifbox` | [crates/services/gifbox](crates/services/gifbox) | Tenor proxy server |  |
| `services/autumn` | [crates/services/autumn](crates/services/autumn) | File server |  |
| `daemons/crond` | [crates/daemons/crond](crates/daemons/crond) | Timed data clean up daemon server |  |
| `daemons/pushd` | [crates/daemons/pushd](crates/daemons/pushd) | Push notification daemon server |  |
@@ -35,22 +37,19 @@ The services and libraries that power the Revolt service.
Rust 1.86.0 or higher.
-> [!CAUTION]
-> The events server has a significant performance regression between Rust 1.77.2 and 1.78.0 onwards, see [issue #341](https://github.com/revoltchat/backend/issues/341). This is currently solved by build time options but we are looking for a proper fix.
-
## Development Guide
-Before contributing, make yourself familiar with [our contribution guidelines](https://developers.revolt.chat/contrib.html) and the [technical documentation for this project](https://revoltchat.github.io/backend/).
+Before contributing, make yourself familiar with [our contribution guidelines](https://developers.stoat.chat/developing/contrib/) and the [technical documentation for this project](https://developers.stoat.chat/).
Before getting started, you'll want to install:
-- Rust toolchain (rustup recommended)
+- mise
- Docker
- Git
- mold (optional, faster compilation)
> A **default.nix** is available for Nix users!
-> Just run `nix-shell` and continue.
+> Run `nix-shell` to activate mise.
As a heads-up, the development environment uses the following ports:
@@ -66,15 +65,25 @@ As a heads-up, the development environment uses the following ports:
| `crates/bonfire` | 14703 |
| `crates/services/autumn` | 14704 |
| `crates/services/january` | 14705 |
+| `crates/services/gifbox` | 14706 |
Now you can clone and build the project:
```bash
-git clone https://github.com/revoltchat/backend revolt-backend
-cd revolt-backend
-cargo build
+git clone https://github.com/stoatchat/stoatchat stoat-backend
+cd stoat-backend
+mise install
+mise build
```
+> [!TIP]
+> You can override `BUILDER` in your `.env` file to run cargo with mold if you installed it:
+>
+> ```bash
+> # .env
+> BUILDER = "mold --run cargo"
+> ```
+
A default configuration `Revolt.toml` is present in this project that is suited for development.
If you'd like to change anything, create a `Revolt.overrides.toml` file and specify relevant variables.
@@ -111,7 +120,7 @@ If you'd like to change anything, create a `Revolt.overrides.toml` file and spec
> - "14672:15672"
> ```
>
-> And corresponding Revolt configuration:
+> With the corresponding Revolt configuration:
>
> ```toml
> # Revolt.overrides.toml
@@ -123,51 +132,41 @@ If you'd like to change anything, create a `Revolt.overrides.toml` file and spec
> [rabbit]
> port = 14072
> ```
+>
+> And mise configuration
+>
+> ```bash
+> #.env
+> DATABASE_PORT = "14017"
+> RABBIT_PORT = "14072"
+> REDIS_PORT = "14079"
+> ```
Then continue:
```bash
-# start other necessary services
-docker compose up -d
+cp livekit.example.yml livekit.yml
-# run everything together
-./scripts/start.sh
-# .. or individually
-# run the API server
-cargo run --bin revolt-delta
-# run the events server
-cargo run --bin revolt-bonfire
-# run the file server
-cargo run --bin revolt-autumn
-# run the proxy server
-cargo run --bin revolt-january
-# run the push daemon (not usually needed in regular development)
-cargo run --bin revolt-pushd
-
-# hint:
-# mold -run
-# mold -run ./scripts/start.sh
+mise start
```
-You can start a web client by doing the following:
+You can start a web client by doing the following in another terminal:
```bash
# if you do not have yarn yet and have a modern Node.js:
corepack enable
# clone the web client and run it:
-git clone --recursive https://github.com/revoltchat/revite
-cd revite
-yarn
-yarn build:deps
-echo "VITE_API_URL=http://local.revolt.chat:14702" > .env.local
-yarn dev --port 14701
+git clone --recursive https://github.com/stoatchat/for-web stoat-web
+cd stoat-web
+# refer to stoat-web/README.md for startup, creating an account and loging in
```
-Then go to http://local.revolt.chat:14701 to create an account/login.
-
When signing up, go to http://localhost:14080 to find confirmation/password reset emails.
+To stop all services, hit (CTRL + c) in the terminal you ran `mise start` and run `mise docker:stop`
+
+
## Deployment Guide
### Cutting new crate releases
@@ -196,14 +195,14 @@ Tag and push a new release by running:
just release
```
-If you have bumped the crate versions, proceed to [GitHub releases](https://github.com/revoltchat/backend/releases/new) to create a changelog.
+If you have bumped the crate versions, proceed to [GitHub releases](https://github.com/stoatchat/stoatchat/releases/new) to create a changelog.
## Testing
First, start the required services:
```sh
-docker compose -f docker-compose.db.yml up -d
+docker compose up -d
```
Now run tests for whichever database:
@@ -215,6 +214,6 @@ TEST_DB=MONGODB cargo nextest run
## License
-The Revolt backend is generally licensed under the [GNU Affero General Public License v3.0](https://github.com/revoltchat/backend/blob/master/LICENSE).
+The Stoat backend is generally licensed under the [GNU Affero General Public License v3.0](https://github.com/stoatchat/stoatchat/blob/main/LICENSE).
**Individual crates may supply their own licenses!**
diff --git a/Revolt.toml b/Revolt.toml
index 33e044348..7398ed53a 100644
--- a/Revolt.toml
+++ b/Revolt.toml
@@ -15,7 +15,7 @@ host = "127.0.0.1"
[hosts]
# Web locations of various services
# Defaults assume all services are reverse-proxied
-# See https://github.com/revoltchat/self-hosted/blob/master/Caddyfile
+# See https://github.com/stoatchat/self-hosted/blob/main/Caddyfile
#
# Remember to change these to https/wss where appropriate in production!
app = "http://local.revolt.chat:14701"
@@ -26,6 +26,11 @@ january = "http://local.revolt.chat:14705"
voso_legacy = ""
voso_legacy_ws = ""
+# Public urls for livekit nodes
+# each entry here should have a corresponding entry under `api.livekit.nodes`
+[hosts.livekit]
+worldwide = "ws://local.revolt.chat:14706"
+
[api]
[api.smtp]
@@ -40,6 +45,18 @@ port = 14025
use_tls = false
use_starttls = false
+[api.livekit]
+
+# Config for livekit nodes
+# Make sure to change the secret when deploying
+# The key and secret should match the values livekit is using
+[api.livekit.nodes.worldwide]
+url = "http://livekit"
+lat = 0.0
+lon = 0.0
+key = "worldwide"
+secret = "ZjCofRlfm6GGtjlifmNpCDkcQbEIIVC0"
+
[files.s3]
# S3 protocol endpoint
endpoint = "http://127.0.0.1:14009"
diff --git a/STYLE_GUIDE.md b/STYLE_GUIDE.md
deleted file mode 100644
index 7999b388a..000000000
--- a/STYLE_GUIDE.md
+++ /dev/null
@@ -1,73 +0,0 @@
-# Code Style Guide
-
-Beyond using Cargo format and Clippy, there are some specific code style guidelines laid out in this document for different parts of the project.
-
-## Writing Style
-
-- Shorten "identifier" to "Id" with that exact casing, i.e. Server Id.
-
-## `core/database` crate
-
-w.r.t. `model.rs` files
-
-- All struct definitions must be commented.
- ```rust
- /// Server
- pub struct Server {
- /// Name of the server
- pub name: String,
- ```
-- Struct definitions should not include derives unless necessary (if additional traits such as Hash are required) and instead use `auto_derived!` and `auto_derived_partial!`.
- ```rust
- auto_derived_partial!(
- /// Server
- pub struct Server { .. },
- "PartialServer"
- );
- ```
-- `auto_derived!` macro accepts multiple entries and should be used as such:
-
- ```rust
- auto_derived!(
- /// Optional fields on server object
- pub enum FieldsServer { .. }
-
- /// Optional fields on server object
- pub enum FieldsRole { .. }
- );
- ```
-
-- If special serialisation conditions are required, such as checking if a boolean is false, use the existing definitions for these functions from the crate root:
- ```rust
- #[serde(skip_serializing_if = "crate::if_false", default)]
- ```
-- `impl` blocks may be defined below the struct definitions and should be ordered in the same order of definition. Methods in the block must follow the same guidelines as traits where-in: methods are ordered in terms of CRUD, there are empty line breaks, and methods are commented.
-
-w.r.t. `ops` module for models
-
-- All traits must use a the name format `AbstractPlural` where Plural is the plural form of the collection. e.g. Servers
-- Traits defined must follow these guidelines:
-
- - Methods are ordered in terms of CRUD, create-read-update-delete ordering.
-
- ```rust
- #[async_trait]
- pub trait AbstractServerMembers: Sync + Send {
- /// Insert a new server member into the database
- async fn insert_member(&self, member: &Member) -> Result<()>;
-
- /// Fetch a server member by their id
- async fn fetch_member(&self, server_id: &str, user_id: &str) -> Result;
-
- /// Update information for a server member
- async fn update_member(&self, .. ) -> Result<()>;
-
- /// Delete a server member by their id
- async fn delete_member(&self, id: &MemberCompositeKey) -> Result<()>;
- }
- ```
-
- - There should be an empty line break between each method declaration.
- - All methods must have an appropriate comment.
-
-- When implementing the trait defined in `ops.rs` with each driver, the method declaration style should be the same for ease of searching: same ordering, same comments, same line breaks.
diff --git a/compose.yml b/compose.yml
index 89b295940..b7e73689d 100644
--- a/compose.yml
+++ b/compose.yml
@@ -8,18 +8,33 @@ services:
# MongoDB
database:
image: mongo
+ command: mongod --replSet rs0
ports:
- "27017:27017"
volumes:
- ./.data/db:/data/db
+ extra_hosts:
+ - "host.docker.internal:host-gateway"
+ healthcheck:
+ test: echo "try { rs.status() } catch (err) { rs.initiate({_id:'rs0',members:[{_id:0,host:'127.0.0.1:27017'}]}) }" | mongosh --port 27017 --quiet
+ interval: 5s
+ timeout: 30s
+ start_period: 0s
+ start_interval: 1s
+ retries: 30
+ ulimits:
+ nofile:
+ soft: 65536
+ hard: 65536
# MinIO
minio:
- image: minio/minio
- command: server /data
+ image: firstfinger/minio:latest
+ #command: server /data
environment:
MINIO_ROOT_USER: minioautumn
MINIO_ROOT_PASSWORD: minioautumn
+ MINIO_REGION: minio
volumes:
- ./.data/minio:/data
ports:
@@ -40,7 +55,7 @@ services:
# Rabbit
rabbit:
- image: rabbitmq:3-management
+ image: rabbitmq:4-management
environment:
RABBITMQ_DEFAULT_USER: rabbituser
RABBITMQ_DEFAULT_PASS: rabbitpass
@@ -55,7 +70,7 @@ services:
# Mock SMTP server
maildev:
- image: soulteary/maildev
+ image: maildev/maildev
ports:
- "14025:25"
- "14080:8080"
@@ -64,3 +79,10 @@ services:
MAILDEV_WEB_PORT: 8080
MAILDEV_INCOMING_USER: smtp
MAILDEV_INCOMING_PASS: smtp
+
+ livekit:
+ image: ghcr.io/stoatchat/livekit-server:v1.9.13
+ command: --config /etc/livekit.yml
+ network_mode: "host"
+ volumes:
+ - ./livekit.yml:/etc/livekit.yml
diff --git a/crates/bonfire/Cargo.toml b/crates/bonfire/Cargo.toml
index e447825d4..b2687266a 100644
--- a/crates/bonfire/Cargo.toml
+++ b/crates/bonfire/Cargo.toml
@@ -1,48 +1,46 @@
[package]
name = "revolt-bonfire"
-version = "0.8.8"
+version = "0.13.7"
license = "AGPL-3.0-or-later"
edition = "2021"
+publish = false
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
# util
-log = "*"
-sentry = "0.31.5"
-lru = "0.7.6"
-ulid = "0.5.0"
-once_cell = "1.9.0"
-redis-kiss = "0.1.4"
-lru_time_cache = "0.11.11"
-async-channel = "2.3.1"
+log = { workspace = true }
+sentry = { workspace = true }
+lru = { workspace = true }
+ulid = { workspace = true }
+once_cell = { workspace = true }
+redis-kiss = { workspace = true }
+lru_time_cache = { workspace = true }
+async-channel = { workspace = true }
# parsing
-querystring = "1.1.0"
+querystring = { workspace = true }
+regex = { workspace = true }
# serde
-bincode = "1.3.3"
-serde_json = "1.0.79"
-rmp-serde = "1.0.0"
-serde = "1.0.136"
+bincode = { workspace = true }
+serde_json = { workspace = true }
+rmp-serde = { workspace = true }
+serde = { workspace = true }
# async
-futures = "0.3.21"
-async-tungstenite = { version = "0.17.0", features = ["async-std-runtime"] }
-async-std = { version = "1.8.0", features = [
- "tokio1",
- "tokio02",
- "attributes",
-] }
+futures = { workspace = true }
+async-tungstenite = { workspace = true, features = ["async-std-runtime"] }
+async-std = { workspace = true }
# core
-authifier = { version = "1.0.15" }
-revolt-result = { path = "../core/result" }
-revolt-models = { path = "../core/models" }
-revolt-config = { path = "../core/config" }
-revolt-database = { path = "../core/database" }
-revolt-permissions = { version = "0.8.8", path = "../core/permissions" }
-revolt-presence = { path = "../core/presence", features = ["redis-is-patched"] }
+authifier = { workspace = true }
+revolt-result = { workspace = true }
+revolt-models = { workspace = true }
+revolt-config = { workspace = true }
+revolt-database = { workspace = true, features = ["voice"] }
+revolt-permissions = { workspace = true }
+revolt-presence = { workspace = true, features = ["redis-is-patched"] }
# redis
-fred = { version = "8.0.1", features = ["subscriber-client"] }
+fred = { workspace = true, features = ["subscriber-client"] }
diff --git a/crates/bonfire/Dockerfile b/crates/bonfire/Dockerfile
index d1829137b..9a608d930 100644
--- a/crates/bonfire/Dockerfile
+++ b/crates/bonfire/Dockerfile
@@ -1,5 +1,5 @@
# Build Stage
-FROM ghcr.io/revoltchat/base:latest AS builder
+FROM ghcr.io/stoatchat/base:latest AS builder
FROM debian:12 AS debian
# Bundle Stage
diff --git a/crates/bonfire/src/config.rs b/crates/bonfire/src/config.rs
index 5642d3baa..d70a63205 100644
--- a/crates/bonfire/src/config.rs
+++ b/crates/bonfire/src/config.rs
@@ -1,9 +1,15 @@
use async_tungstenite::tungstenite::{handshake, Message};
use futures::channel::oneshot::Sender;
+use once_cell::sync::Lazy;
+use regex::Regex;
use revolt_database::events::client::ReadyPayloadFields;
use revolt_result::{create_error, Result};
use serde::{Deserialize, Serialize};
+/// matches either a single word ie "users" or a key and value ie "settings[notifications]"
+static READY_PAYLOAD_FIELD_REGEX: Lazy =
+ Lazy::new(|| Regex::new(r#"^(\w+)(?:\[(\S+)\])?$"#).unwrap());
+
/// Enumeration of supported protocol formats
#[derive(Debug)]
pub enum ProtocolFormat {
@@ -17,6 +23,7 @@ pub struct ProtocolConfiguration {
protocol_version: i32,
format: ProtocolFormat,
session_token: Option,
+ ready_payload_fields: ReadyPayloadFields,
}
impl ProtocolConfiguration {
@@ -25,11 +32,13 @@ impl ProtocolConfiguration {
protocol_version: i32,
format: ProtocolFormat,
session_token: Option,
+ ready_payload_fields: ReadyPayloadFields,
) -> Self {
Self {
protocol_version,
format,
session_token,
+ ready_payload_fields,
}
}
@@ -86,14 +95,8 @@ impl ProtocolConfiguration {
}
/// Get ready payload fields
- pub fn get_ready_payload_fields(&self) -> Vec {
- vec![
- ReadyPayloadFields::Users,
- ReadyPayloadFields::Servers,
- ReadyPayloadFields::Channels,
- ReadyPayloadFields::Members,
- ReadyPayloadFields::Emoji,
- ]
+ pub fn get_ready_payload_fields(&self) -> &ReadyPayloadFields {
+ &self.ready_payload_fields
}
}
@@ -124,6 +127,23 @@ impl handshake::server::Callback for WebsocketHandshakeCallback {
let mut protocol_version = 1;
let mut format = ProtocolFormat::Json;
let mut session_token = None;
+ let mut ready_payload_fields = if params.iter().any(|(k, _)| *k == "ready") {
+ // If they pass the ready field, set all fields to false
+
+ ReadyPayloadFields {
+ users: false,
+ servers: false,
+ channels: false,
+ members: false,
+ emojis: false,
+ voice_states: false,
+ user_settings: Vec::new(),
+ channel_unreads: false,
+ policy_changes: false,
+ }
+ } else {
+ ReadyPayloadFields::default()
+ };
// Parse and map parameters from key-value to known variables.
for (key, value) in params {
@@ -139,6 +159,31 @@ impl handshake::server::Callback for WebsocketHandshakeCallback {
_ => {}
},
"token" => session_token = Some(value.into()),
+ "ready" => {
+ // Re-enable all the fields the client specifies
+ if let Some(captures) = READY_PAYLOAD_FIELD_REGEX.captures(value) {
+ if let Some(field) = captures.get(0) {
+ match field.as_str() {
+ "users" => ready_payload_fields.users = true,
+ "servers" => ready_payload_fields.servers = true,
+ "channels" => ready_payload_fields.channels = true,
+ "members" => ready_payload_fields.members = true,
+ "emojis" => ready_payload_fields.emojis = true,
+ "voice_states" => ready_payload_fields.voice_states = true,
+ "channel_unreads" => ready_payload_fields.channel_unreads = true,
+ "user_settings" => {
+ if let Some(subkey) = captures.get(1) {
+ ready_payload_fields
+ .user_settings
+ .push(subkey.as_str().to_string());
+ }
+ }
+ "policy_changes" => ready_payload_fields.policy_changes = true,
+ _ => {}
+ }
+ }
+ }
+ }
_ => {}
}
}
@@ -151,6 +196,7 @@ impl handshake::server::Callback for WebsocketHandshakeCallback {
protocol_version,
format,
session_token,
+ ready_payload_fields,
})
.is_ok()
{
diff --git a/crates/bonfire/src/events/impl.rs b/crates/bonfire/src/events/impl.rs
index 160e7c6c0..96a0f2ab0 100644
--- a/crates/bonfire/src/events/impl.rs
+++ b/crates/bonfire/src/events/impl.rs
@@ -1,9 +1,11 @@
-use std::collections::HashSet;
+use std::collections::{HashMap, HashSet};
use futures::future::join_all;
+use redis_kiss::AsyncCommands;
use revolt_database::{
events::client::{EventV1, ReadyPayloadFields},
util::permissions::DatabasePermissionQuery,
+ voice::{get_channel_voice_state, UserVoiceChannel},
Channel, Database, Member, MemberCompositeKey, Presence, RelationshipStatus,
};
use revolt_models::v0;
@@ -17,8 +19,9 @@ use super::state::{Cache, State};
impl Cache {
/// Check whether the current user can view a channel
pub async fn can_view_channel(&self, db: &Database, channel: &Channel) -> bool {
+ #[allow(deprecated)]
match &channel {
- Channel::TextChannel { server, .. } | Channel::VoiceChannel { server, .. } => {
+ Channel::TextChannel { server, .. } => {
let member = self.members.get(server);
let server = self.servers.get(server);
let mut query =
@@ -95,21 +98,23 @@ impl State {
pub async fn generate_ready_payload(
&mut self,
db: &Database,
- fields: Vec,
+ fields: &ReadyPayloadFields,
) -> Result {
let user = self.clone_user();
self.cache.is_bot = user.bot.is_some();
// Fetch pending policy changes.
- let policy_changes = if user.bot.is_some() {
- vec![]
+ let policy_changes = if user.bot.is_some() || !fields.policy_changes {
+ None
} else {
- db.fetch_policy_changes()
- .await?
- .into_iter()
- .filter(|policy| policy.created_time > user.last_acknowledged_policy_change)
- .map(Into::into)
- .collect()
+ Some(
+ db.fetch_policy_changes()
+ .await?
+ .into_iter()
+ .filter(|policy| policy.created_time > user.last_acknowledged_policy_change)
+ .map(Into::into)
+ .collect(),
+ )
};
// Find all relationships to the user.
@@ -120,12 +125,7 @@ impl State {
.unwrap_or_default();
// Fetch all memberships with their corresponding servers.
- let members: Vec = db.fetch_all_memberships(&user.id).await?;
- self.cache.members = members
- .iter()
- .cloned()
- .map(|x| (x.id.server.clone(), x))
- .collect();
+ let mut members: Vec = db.fetch_all_memberships(&user.id).await?;
let server_ids: Vec = members.iter().map(|x| x.id.server.clone()).collect();
let servers = db.fetch_servers(&server_ids).await?;
@@ -154,6 +154,55 @@ impl State {
}
}
+ let voice_states = if fields.voice_states {
+ let mut voice_state_server_members: HashMap> = HashMap::new();
+
+ // fetch voice states for all the channels we can see
+ let mut voice_states = Vec::new();
+
+ for channel in channels.iter().filter(|c| {
+ matches!(
+ c,
+ Channel::DirectMessage { .. }
+ | Channel::Group { .. }
+ | Channel::TextChannel { voice: Some(_), .. }
+ )
+ }) {
+ if let Ok(Some(voice_state)) =
+ get_channel_voice_state(&UserVoiceChannel::from_channel(channel)).await
+ {
+ if let Some(server) = channel.server() {
+ let set = voice_state_server_members
+ .entry(server.to_string())
+ .or_default();
+
+ for participant in &voice_state.participants {
+ user_ids.insert(participant.id.clone());
+ set.insert(participant.id.clone());
+ }
+ } else {
+ for participant in &voice_state.participants {
+ user_ids.insert(participant.id.clone());
+ }
+ }
+
+ voice_states.push(voice_state);
+ }
+ }
+
+ // Fetch all the members for for the participants who are in a server
+ for (server, user_ids) in voice_state_server_members {
+ let user_ids = user_ids.into_iter().collect::>();
+ let voice_members = db.fetch_members(&server, &user_ids).await?;
+
+ members.extend(voice_members);
+ }
+
+ Some(voice_states)
+ } else {
+ None
+ };
+
// Fetch presence data for known users.
let online_ids = filter_online(&user_ids.iter().cloned().collect::>()).await;
@@ -167,8 +216,14 @@ impl State {
)
.await?;
+ self.cache.members = members
+ .iter()
+ .cloned()
+ .map(|x| (x.id.server.clone(), x))
+ .collect();
+
// Fetch customisations.
- let emojis = if fields.contains(&ReadyPayloadFields::Emoji) {
+ let emojis = if fields.emojis {
Some(
db.fetch_emoji_by_parent_ids(
&servers
@@ -176,25 +231,34 @@ impl State {
.map(|x| x.id.to_string())
.collect::>(),
)
- .await?,
+ .await?
+ .into_iter()
+ .map(|emoji| emoji.into())
+ .collect(),
)
} else {
None
};
// Fetch user settings
- let user_settings = if let Some(ReadyPayloadFields::UserSettings(keys)) = fields
- .iter()
- .find(|e| matches!(e, ReadyPayloadFields::UserSettings(_)))
- {
- Some(db.fetch_user_settings(&user.id, keys).await?)
+ let user_settings = if !fields.user_settings.is_empty() {
+ Some(
+ db.fetch_user_settings(&user.id, &fields.user_settings)
+ .await?,
+ )
} else {
None
};
// Fetch channel unreads
- let channel_unreads = if fields.contains(&ReadyPayloadFields::ChannelUnreads) {
- Some(db.fetch_unreads(&user.id).await?)
+ let channel_unreads = if fields.channel_unreads {
+ Some(
+ db.fetch_unreads(&user.id)
+ .await?
+ .into_iter()
+ .map(|unread| unread.into())
+ .collect(),
+ )
} else {
None
};
@@ -241,30 +305,27 @@ impl State {
}
Ok(EventV1::Ready {
- users: if fields.contains(&ReadyPayloadFields::Users) {
- Some(users)
- } else {
- None
- },
- servers: if fields.contains(&ReadyPayloadFields::Servers) {
+ users: if fields.users { Some(users) } else { None },
+ servers: if fields.servers {
Some(servers.into_iter().map(Into::into).collect())
} else {
None
},
- channels: if fields.contains(&ReadyPayloadFields::Channels) {
+ channels: if fields.channels {
Some(channels.into_iter().map(Into::into).collect())
} else {
None
},
- members: if fields.contains(&ReadyPayloadFields::Members) {
+ members: if fields.members {
Some(members.into_iter().map(Into::into).collect())
} else {
None
},
- emojis: emojis.map(|vec| vec.into_iter().map(Into::into).collect()),
+ voice_states,
+ emojis,
user_settings,
- channel_unreads: channel_unreads.map(|vec| vec.into_iter().map(Into::into).collect()),
+ channel_unreads,
policy_changes,
})
@@ -279,19 +340,14 @@ impl State {
let id = &id.to_string();
for (channel_id, channel) in &self.cache.channels {
- match channel {
- Channel::TextChannel { server, .. } | Channel::VoiceChannel { server, .. } => {
- if server == id {
- channel_ids.insert(channel_id.clone());
-
- if self.cache.can_view_channel(db, channel).await {
- added_channels.push(channel_id.clone());
- } else {
- removed_channels.push(channel_id.clone());
- }
- }
+ if channel.server() == Some(id) {
+ channel_ids.insert(channel_id.clone());
+
+ if self.cache.can_view_channel(db, channel).await {
+ added_channels.push(channel_id.clone());
+ } else {
+ removed_channels.push(channel_id.clone());
}
- _ => {}
}
}
@@ -346,6 +402,11 @@ impl State {
/// Push presence change to the user and all associated server topics
pub async fn broadcast_presence_change(&self, target: bool) {
+ let config = revolt_config::config().await;
+ if config.disable_events_dont_use {
+ return;
+ }
+
if if let Some(status) = &self.cache.users.get(&self.cache.user_id).unwrap().status {
status.presence != Some(Presence::Invisible)
} else {
@@ -459,6 +520,7 @@ impl State {
server,
channels,
emojis: _,
+ voice_states: _,
} => {
self.insert_subscription(id.clone()).await;
diff --git a/crates/bonfire/src/events/state.rs b/crates/bonfire/src/events/state.rs
index eae398019..daa22269a 100644
--- a/crates/bonfire/src/events/state.rs
+++ b/crates/bonfire/src/events/state.rs
@@ -1,7 +1,5 @@
use std::{
- collections::{HashMap, HashSet},
- sync::Arc,
- time::Duration,
+ collections::{HashMap, HashSet}, num::NonZeroUsize, sync::Arc, time::Duration
};
use async_std::sync::{Mutex, RwLock};
@@ -57,7 +55,7 @@ impl Default for Cache {
members: Default::default(),
servers: Default::default(),
- seen_events: LruCache::new(20),
+ seen_events: LruCache::new(NonZeroUsize::new(20).unwrap()),
}
}
}
diff --git a/crates/bonfire/src/websocket.rs b/crates/bonfire/src/websocket.rs
index 913245e9c..ee5512896 100644
--- a/crates/bonfire/src/websocket.rs
+++ b/crates/bonfire/src/websocket.rs
@@ -5,7 +5,7 @@ use authifier::AuthifierEvent;
use fred::{
error::RedisErrorKind,
interfaces::{ClientLike, EventInterface, PubsubInterface},
- types::RedisConfig,
+ types::{ReconnectPolicy, RedisConfig},
};
use futures::{
channel::oneshot,
@@ -13,7 +13,7 @@ use futures::{
stream::{SplitSink, SplitStream},
FutureExt, SinkExt, StreamExt, TryStreamExt,
};
-use redis_kiss::{PayloadType, REDIS_PAYLOAD_TYPE, REDIS_URI};
+use redis_kiss::{get_connection, AsyncCommands, PayloadType, REDIS_PAYLOAD_TYPE, REDIS_URI};
use revolt_config::report_internal_error;
use revolt_database::{
events::{client::EventV1, server::ClientMessage},
@@ -32,6 +32,7 @@ use sentry::Level;
use crate::config::{ProtocolConfiguration, WebsocketHandshakeCallback};
use crate::events::state::{State, SubscriptionStateChange};
+use revolt_models::v0;
type WsReader = SplitStream>;
type WsWriter = SplitSink, async_tungstenite::tungstenite::Message>;
@@ -128,6 +129,14 @@ pub async fn client(db: &'static Database, stream: TcpStream, addr: SocketAddr)
return;
}
+ let slowmodes = fetch_user_slowmodes(&user_id).await.unwrap_or_default();
+ if !slowmodes.is_empty() {
+ let event = EventV1::UserSlowmodes { slowmodes };
+ if report_internal_error!(write.send(config.encode(&event)).await).is_err() {
+ return;
+ }
+ }
+
// Create presence session.
let (first_session, session_id) = create_session(&user_id, 0).await;
@@ -218,10 +227,16 @@ async fn listener(
kill_signal_r: async_channel::Receiver<()>,
write: &Mutex,
) {
- let redis_config = RedisConfig::from_url(&REDIS_URI).unwrap();
- let subscriber = match report_internal_error!(
- fred::types::Builder::from_config(redis_config).build_subscriber_client()
- ) {
+ let stoat_config = revolt_config::config().await;
+ let url = stoat_config
+ .database
+ .redis_pubsub
+ .unwrap_or(REDIS_URI.to_string());
+
+ let redis_config = RedisConfig::from_url(&url).unwrap();
+ let mut builder = fred::types::Builder::from_config(redis_config);
+ builder.set_policy(ReconnectPolicy::new_exponential(8, 100, 30_000, 2));
+ let subscriber = match report_internal_error!(builder.build_subscriber_client()) {
Ok(subscriber) => subscriber,
Err(_) => return,
};
@@ -230,16 +245,21 @@ async fn listener(
return;
}
+ // Let Fred automatically re-subscribe to tracked channels on reconnect.
+ subscriber.manage_subscriptions();
+
// Handle Redis connection dropping
let (clean_up_s, clean_up_r) = async_channel::bounded(1);
let clean_up_s = Arc::new(Mutex::new(clean_up_s));
subscriber.on_error(move |err| {
+ warn!("Redis subscriber error: {:?}", err);
if let RedisErrorKind::Canceled = err.kind() {
let clean_up_s = clean_up_s.clone();
spawn(async move {
clean_up_s.lock().await.send(()).await.ok();
});
}
+ // Transient errors (IO, timeout) are handled by the reconnect policy.
Ok(())
});
@@ -422,6 +442,8 @@ async fn worker(
mut read: WsReader,
write: &Mutex,
) {
+ let revolt_config = revolt_config::config().await;
+
loop {
let t1 = read.try_next().fuse();
let t2 = kill_signal_r.recv().fuse();
@@ -458,6 +480,10 @@ async fn worker(
match payload {
ClientMessage::BeginTyping { channel } => {
+ if revolt_config.disable_events_dont_use {
+ continue;
+ }
+
if !subscribed.read().await.contains(&channel) {
continue;
}
@@ -470,6 +496,10 @@ async fn worker(
.await;
}
ClientMessage::EndTyping { channel } => {
+ if revolt_config.disable_events_dont_use {
+ continue;
+ }
+
if !subscribed.read().await.contains(&channel) {
continue;
}
@@ -507,3 +537,42 @@ async fn worker(
}
}
}
+
+async fn fetch_user_slowmodes(user_id: &str) -> Option> {
+ let mut conn = get_connection().await.ok()?.into_inner();
+ let idx_key = format!("slowmode_idx:{}", user_id);
+
+ let channel_ids: Vec = conn.smembers(&idx_key).await.unwrap_or_default();
+ if channel_ids.is_empty() {
+ return Some(vec![]);
+ }
+
+ // Bulk fetch all TTLs in one round trip
+ let mut pipe = redis_kiss::redis::pipe();
+ for channel_id in &channel_ids {
+ pipe.ttl(format!("slowmode:{}:{}", user_id, channel_id));
+ }
+ let ttls: Vec = pipe.query_async(&mut conn).await.unwrap_or_default();
+
+ // Partition into alive/expired in one pass
+ let mut slowmodes = vec![];
+ let mut expired = vec![];
+ for (channel_id, ttl) in channel_ids.iter().zip(ttls.iter()) {
+ if *ttl > 0 {
+ slowmodes.push(v0::ChannelSlowmode {
+ channel_id: channel_id.clone(),
+ duration: *ttl as u64,
+ retry_after: *ttl as u64,
+ });
+ } else {
+ expired.push(channel_id.as_str());
+ }
+ }
+
+ // Bulk remove all expired members in one SREM call
+ if !expired.is_empty() {
+ conn.srem::<_, _, ()>(&idx_key, expired).await.ok();
+ }
+
+ Some(slowmodes)
+}
diff --git a/crates/core/coalesced/Cargo.toml b/crates/core/coalesced/Cargo.toml
new file mode 100644
index 000000000..1c8102adb
--- /dev/null
+++ b/crates/core/coalesced/Cargo.toml
@@ -0,0 +1,28 @@
+[package]
+name = "revolt-coalesced"
+version = "0.13.7"
+edition = "2021"
+license = "MIT"
+authors = ["Paul Makles ", "Zomatree "]
+description = "Revolt Backend: Coalescion service"
+repository = "https://github.com/stoatchat/stoatchat"
+
+[features]
+tokio = ["dep:tokio"]
+queue = ["dep:indexmap"]
+cache = ["dep:lru"]
+
+default = ["tokio"]
+
+[dependencies]
+tokio = { workspace = true, features = ["sync"], optional = true }
+indexmap = { workspace = true, optional = true }
+lru = { workspace = true, optional = true }
+
+[dev-dependencies]
+tokio = { workspace = true, features = [
+ "rt",
+ "rt-multi-thread",
+ "macros",
+ "time",
+] }
diff --git a/crates/core/coalesced/LICENSE b/crates/core/coalesced/LICENSE
new file mode 100644
index 000000000..7c2815b9f
--- /dev/null
+++ b/crates/core/coalesced/LICENSE
@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) 2024 Pawel Makles
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
diff --git a/crates/core/coalesced/src/config.rs b/crates/core/coalesced/src/config.rs
new file mode 100644
index 000000000..23da16989
--- /dev/null
+++ b/crates/core/coalesced/src/config.rs
@@ -0,0 +1,24 @@
+#[derive(Clone, PartialEq, Eq, Debug)]
+/// Config values for [`CoalescionService`].
+pub struct CoalescionServiceConfig {
+ /// How many tasks are running at once
+ pub max_concurrent: Option,
+ /// Whether to queue tasks once `max_concurrent` is reached
+ #[cfg(feature = "queue")]
+ pub queue_requests: bool,
+ /// Max amount of tasks in the buffer queue
+ #[cfg(feature = "queue")]
+ pub max_queue: Option,
+}
+
+impl Default for CoalescionServiceConfig {
+ fn default() -> Self {
+ Self {
+ max_concurrent: Some(100),
+ #[cfg(feature = "queue")]
+ queue_requests: true,
+ #[cfg(feature = "queue")]
+ max_queue: Some(100)
+ }
+ }
+}
diff --git a/crates/core/coalesced/src/error.rs b/crates/core/coalesced/src/error.rs
new file mode 100644
index 000000000..6be897858
--- /dev/null
+++ b/crates/core/coalesced/src/error.rs
@@ -0,0 +1,27 @@
+use std::fmt;
+
+#[derive(Clone, Copy, PartialEq, Eq, Debug, Hash)]
+/// Coalescion service error.
+pub enum Error {
+ /// Failed to receive the actions return from the channel for unknown reason
+ RecvError,
+ /// Reached the `max_concurrent` amount of actions running at once and could not queue the action
+ MaxConcurrent,
+ /// Reached the `max_queue` amount of actions in the queue
+ MaxQueue,
+ /// Failed to downcast the type to the current type being returned, this will be most likely an ID collision
+ DowncastError,
+}
+
+impl fmt::Display for Error {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ match self {
+ Error::RecvError => write!(f, "Unable to receive data from the channel"),
+ Error::MaxConcurrent => write!(f, "Max number of tasks running at once"),
+ Error::MaxQueue => write!(f, "Max number of tasks in queue"),
+ Error::DowncastError => write!(f, "Failed to downcast type, possible key collision with different types")
+ }
+ }
+}
+
+impl std::error::Error for Error {}
diff --git a/crates/core/coalesced/src/lib.rs b/crates/core/coalesced/src/lib.rs
new file mode 100644
index 000000000..89839e23a
--- /dev/null
+++ b/crates/core/coalesced/src/lib.rs
@@ -0,0 +1,39 @@
+//! # Coalesced
+//!
+//! Coalescion service to group, caching and queue duplicate actions.
+//! useful for deduplicating web requests, database lookups and other similar resource
+//! intensive or rate-limited actions.
+//!
+//! ## Features
+//! - `tokio`: Uses tokio for the async backend, this is currently the only backend.
+//! - `queue`: Whether to support queueing requests to only allow X amount of actions running at once.
+//! - `cache`: Whether to cache the actions results for future actions with the same id, uses an LRU cache internally.
+//!
+//! [`CoalescionService`] uses both [`Arc`] and [`RwLock`] internally and can be cheaply cloned to
+//! use in your codebase.
+//!
+//! It is common practice to wrap the service and in your own which delegates the executions to ensure all ids are tracked in one location across your codebase.
+//!
+//! All values are stored using [`Any`] and must be [`'static`] + [`Send`] + [`Sync`], if there is an id mismatch
+//! and a type is wrong the library will return an error, values returned from the service are also
+//! wrapped in an [`Arc`] as they are shared to each duplicate action.
+//!
+//! ## Example:
+//! ```rs
+//! use revolt_coalesced::CoalescionService;
+//!
+//! let service = CoalescionService::new();
+//!
+//! let user_id = "my_user_id";
+//! let user = service.execute(user_id, || async move {
+//! database.fetch_user(user_id).await.unwrap()
+//! }).await;
+//! ```
+
+mod config;
+mod error;
+mod service;
+
+pub use config::CoalescionServiceConfig;
+pub use error::Error;
+pub use service::CoalescionService;
diff --git a/crates/core/coalesced/src/service.rs b/crates/core/coalesced/src/service.rs
new file mode 100644
index 000000000..aadadc8a0
--- /dev/null
+++ b/crates/core/coalesced/src/service.rs
@@ -0,0 +1,208 @@
+use std::{any::Any, collections::HashMap, fmt::Debug, future::Future, hash::Hash, sync::Arc};
+
+use tokio::sync::{
+ watch::{channel as watch_channel, Receiver},
+ RwLock,
+};
+
+#[cfg(feature = "cache")]
+use lru::LruCache;
+
+#[cfg(feature = "queue")]
+use indexmap::IndexMap;
+
+use crate::{CoalescionServiceConfig, Error};
+
+#[derive(Debug, Clone)]
+#[allow(clippy::type_complexity)]
+/// # Coalescion service
+///
+/// See module description for example usage.
+pub struct CoalescionService {
+ config: Arc,
+ watchers: Arc, Error>>>>>>,
+ #[cfg(feature = "queue")]
+ queue: Arc, Error>>>>>>,
+ #[cfg(feature = "cache")]
+ cache: Option>>>>,
+}
+
+impl CoalescionService {
+ pub fn new() -> Self {
+ Default::default()
+ }
+
+ pub fn from_config(config: CoalescionServiceConfig) -> Self {
+ Self {
+ config: Arc::new(config),
+ watchers: Arc::new(RwLock::new(HashMap::new())),
+ #[cfg(feature = "queue")]
+ queue: Arc::new(RwLock::new(IndexMap::new())),
+ #[cfg(feature = "cache")]
+ cache: None,
+ }
+ }
+
+ #[cfg(feature = "cache")]
+ pub fn from_cache(
+ config: CoalescionServiceConfig,
+ cache: LruCache>,
+ ) -> Self {
+ Self {
+ cache: Some(Arc::new(Mutex::new(cache))),
+ ..Self::from_config(config)
+ }
+ }
+
+ async fn wait_for(
+ &self,
+ mut receiver: Receiver, Error>>>,
+ ) -> Result, Error> {
+ receiver
+ .wait_for(|v| v.is_some())
+ .await
+ .map_err(|_| Error::RecvError)
+ .and_then(|r| r.clone().unwrap())
+ .and_then(|arc| Arc::downcast(arc).map_err(|_| Error::DowncastError))
+ }
+
+ async fn insert_and_execute<
+ Value: Send + Sync + 'static,
+ F: FnOnce() -> Fut,
+ Fut: Future
diff --git a/crates/core/database/templates/deletion.original.html b/crates/core/database/templates/deletion.original.html
index 0da330eac..e3ad4e508 100644
--- a/crates/core/database/templates/deletion.original.html
+++ b/crates/core/database/templates/deletion.original.html
@@ -1,38 +1,37 @@
-
-
-
-
-
-
-
-
-
Account Deletion
-
- You requested to have your account deleted, if you did not perform
- this action please take measures to secure your account immediately.
-
-
Confirm
-
-
- This email is intended for {{email}}
- Sent from Revolt
- Made in Europe
-
-
- Revolt Platforms Ltd. is a company incorporated and registered under the
- laws of England and Wales.
- Registered Company Number: 16260658
- Registered Office:
- Suite 5703 Unit 3A, 34-35 Hatton Garden,
- Holborn, United Kingdom, EC1N 8DX
-
+
+
+
+
+
+
+
+
+
+
+
Account Deletion
+
+ You requested to have your account deleted, if you did not perform
+ this action please take measures to secure your account immediately.
+
+
Confirm
-
-
+
+ This email is intended for {{email}}
+ Sent from Stoat
+ Made in Europe
+
+
+ Revolt Platforms Ltd. is a company incorporated and registered under the
+ laws of England and Wales.
+ Registered Company Number: 16260658
+ Registered Office:
+ Suite 5703 Unit 3A, 34-35 Hatton Garden,
+ Holborn, United Kingdom, EC1N 8DX
+
+
+
+
+
\ No newline at end of file
diff --git a/crates/core/database/templates/deletion.txt b/crates/core/database/templates/deletion.txt
index b6c2871b0..648059a17 100644
--- a/crates/core/database/templates/deletion.txt
+++ b/crates/core/database/templates/deletion.txt
@@ -3,7 +3,7 @@ You requested to have your account deleted, if you did not perform this action p
Please navigate to: {{url}}
This email is intended for {{email}}
-Sent by Revolt
+Sent by Stoat
Made in Europe
Revolt Platforms Ltd. is a company incorporated and registered under the laws of England and Wales.
diff --git a/crates/core/database/templates/deletion.whitelabel.txt b/crates/core/database/templates/deletion.whitelabel.txt
index 491280aac..b040a51ed 100644
--- a/crates/core/database/templates/deletion.whitelabel.txt
+++ b/crates/core/database/templates/deletion.whitelabel.txt
@@ -4,6 +4,6 @@ Please navigate to: {{url}}
This email is intended for {{email}}
-This email has no association with Revolt or Revolt Platforms Ltd.
+This email has no association with Stoat or Revolt Platforms Ltd.
Learn more about third party instances here:
-https://developers.revolt.chat/faq.html
+https://developers.stoat.chat/faq/
diff --git a/crates/core/database/templates/reset-existing.html b/crates/core/database/templates/reset-existing.html
index 00ef782b6..1ed6424d3 100644
--- a/crates/core/database/templates/reset-existing.html
+++ b/crates/core/database/templates/reset-existing.html
@@ -1,7 +1,7 @@
-
+
@@ -42,7 +42,7 @@
|
-
+
|
@@ -120,7 +120,7 @@
This email is intended for {{email}}
- Sent from Revolt
+ Sent from Stoat
Made in Europe
diff --git a/crates/core/database/templates/reset-existing.original.html b/crates/core/database/templates/reset-existing.original.html
index 1273a69c4..a68c3d35d 100644
--- a/crates/core/database/templates/reset-existing.original.html
+++ b/crates/core/database/templates/reset-existing.original.html
@@ -9,7 +9,8 @@
Password Reset
@@ -25,7 +26,7 @@ Password Reset
This email is intended for {{email}}
- Sent from Revolt
+ Sent from Stoat
Made in Europe
diff --git a/crates/core/database/templates/reset-existing.txt b/crates/core/database/templates/reset-existing.txt
index 8c8fc9edc..6637f9086 100644
--- a/crates/core/database/templates/reset-existing.txt
+++ b/crates/core/database/templates/reset-existing.txt
@@ -8,7 +8,7 @@ password on it, click below to continue.
Please navigate to: {{url}}
This email is intended for {{email}}
-Sent by Revolt
+Sent by Stoat
Made in Europe
Revolt Platforms Ltd. is a company incorporated and registered under the laws of England and Wales.
diff --git a/crates/core/database/templates/reset.html b/crates/core/database/templates/reset.html
index 9b938f7a4..b2f05cf2e 100644
--- a/crates/core/database/templates/reset.html
+++ b/crates/core/database/templates/reset.html
@@ -1,7 +1,7 @@
-
+
@@ -42,7 +42,7 @@
|
-
+
|
@@ -104,7 +104,7 @@
This email is intended for {{email}}
- Sent from Revolt
+ Sent from Stoat
Made in Europe
diff --git a/crates/core/database/templates/reset.original.html b/crates/core/database/templates/reset.original.html
index f15d03289..76b7b5c6e 100644
--- a/crates/core/database/templates/reset.original.html
+++ b/crates/core/database/templates/reset.original.html
@@ -1,34 +1,34 @@
-
-
-
-
-
-
-
-
-
Password Reset
-
You requested a password reset, click below to continue.
-
Reset
-
-
- This email is intended for {{email}}
- Sent from Revolt
- Made in Europe
-
-
- Revolt Platforms Ltd. is a company incorporated and registered under the
- laws of England and Wales.
- Registered Company Number: 16260658
- Registered Office:
- Suite 5703 Unit 3A, 34-35 Hatton Garden,
- Holborn, United Kingdom, EC1N 8DX
-
+
+
+
+
+
+
+
+
+
+
+
Password Reset
+
You requested a password reset, click below to continue.
+
Reset
-
-
+
+ This email is intended for {{email}}
+ Sent from Stoat
+ Made in Europe
+
+
+ Revolt Platforms Ltd. is a company incorporated and registered under the
+ laws of England and Wales.
+ Registered Company Number: 16260658
+ Registered Office:
+ Suite 5703 Unit 3A, 34-35 Hatton Garden,
+ Holborn, United Kingdom, EC1N 8DX
+
+
+
+
+
\ No newline at end of file
diff --git a/crates/core/database/templates/reset.txt b/crates/core/database/templates/reset.txt
index 020032c71..266a3e2ed 100644
--- a/crates/core/database/templates/reset.txt
+++ b/crates/core/database/templates/reset.txt
@@ -3,7 +3,7 @@ You requested a password reset, if you did not perform this action you can safel
Please navigate to: {{url}}
This email is intended for {{email}}
-Sent by Revolt
+Sent by Stoat
Made in Europe
Revolt Platforms Ltd. is a company incorporated and registered under the laws of England and Wales.
diff --git a/crates/core/database/templates/reset.whitelabel.txt b/crates/core/database/templates/reset.whitelabel.txt
index 9ac5028fc..10e8ca2d2 100644
--- a/crates/core/database/templates/reset.whitelabel.txt
+++ b/crates/core/database/templates/reset.whitelabel.txt
@@ -4,6 +4,6 @@ Please navigate to: {{url}}
This email is intended for {{email}}
-This email has no association with Revolt or Revolt Platforms Ltd.
+This email has no association with Stoat or Revolt Platforms Ltd.
Learn more about third party instances here:
-https://developers.revolt.chat/faq.html
+https://developers.stoat.chat/faq/
diff --git a/crates/core/database/templates/suspension.html b/crates/core/database/templates/suspension.html
index 5002c2580..27b7331e7 100644
--- a/crates/core/database/templates/suspension.html
+++ b/crates/core/database/templates/suspension.html
@@ -1,7 +1,7 @@
-
+
@@ -42,7 +42,7 @@
|
-
+
|
@@ -107,7 +107,7 @@
Further violations may result in a permanent ban depending on
severity, please abide by the
- Acceptable Usage Policy.
+ Community Guidelines.
@@ -134,7 +134,7 @@
This email is intended for {{email}}
- Sent from Revolt
+ Sent from Stoat
Made in Europe
diff --git a/crates/core/database/templates/suspension.original.html b/crates/core/database/templates/suspension.original.html
index fbbbb4168..de49512d2 100644
--- a/crates/core/database/templates/suspension.original.html
+++ b/crates/core/database/templates/suspension.original.html
@@ -1,46 +1,45 @@
-
-
-
-
-
-
-
-
-
Account Suspended
-
Your account has been suspended, for one or more reasons:
-
-
- You will be able to use your account again in {{duration}} days.
-
-
- Further violations may result in a permanent ban depending on
- severity, please abide by the
- Acceptable Usage Policy.
-
-
Ban evasion is prohibited and will be dealt with accordingly.
-
-
- This email is intended for {{email}}
- Sent from Revolt
- Made in Europe
-
-
- Revolt Platforms Ltd. is a company incorporated and registered under the
- laws of England and Wales.
- Registered Company Number: 16260658
- Registered Office:
- Suite 5703 Unit 3A, 34-35 Hatton Garden,
- Holborn, United Kingdom, EC1N 8DX
-
+
+
+
+
+
+
+
+
+
+
+
Account Suspended
+
Your account has been suspended, for one or more reasons:
+
+
+ You will be able to use your account again in {{duration}} days.
+
+
+ Further violations may result in a permanent ban depending on
+ severity, please abide by the
+ Acceptable Usage Policy.
+
+
Ban evasion is prohibited and will be dealt with accordingly.
-
-
+
+ This email is intended for {{email}}
+ Sent from Stoat
+ Made in Europe
+
+
+ Revolt Platforms Ltd. is a company incorporated and registered under the
+ laws of England and Wales.
+ Registered Company Number: 16260658
+ Registered Office:
+ Suite 5703 Unit 3A, 34-35 Hatton Garden,
+ Holborn, United Kingdom, EC1N 8DX
+
+
+
+
+
\ No newline at end of file
diff --git a/crates/core/database/templates/suspension.txt b/crates/core/database/templates/suspension.txt
index 30d48d846..293f8a0c2 100644
--- a/crates/core/database/templates/suspension.txt
+++ b/crates/core/database/templates/suspension.txt
@@ -3,12 +3,12 @@ Your account has been suspended, for one or more reasons:
You will be able to use your account again in {{duration}} days.
-Further violations may result in a permanent ban depending on severity, please abide by the Acceptable Usage Policy (https://revolt.chat/aup).
+Further violations may result in a permanent ban depending on severity, please abide by the Community Guidelines (https://stoat.chat/legal/community-guidelines).
Ban evasion is prohibited and will be dealt with accordingly.
This email is intended for {{email}}
-Sent by Revolt
+Sent by Stoat
Made in Europe
Revolt Platforms Ltd. is a company incorporated and registered under the laws of England and Wales.
diff --git a/crates/core/database/templates/suspension.whitelabel.txt b/crates/core/database/templates/suspension.whitelabel.txt
index 45d9aa560..bd82145d4 100644
--- a/crates/core/database/templates/suspension.whitelabel.txt
+++ b/crates/core/database/templates/suspension.whitelabel.txt
@@ -3,6 +3,6 @@ Your account has been suspended, for one or more reasons:
This email is intended for {{email}}
-This email has no association with Revolt or Revolt Platforms Ltd.
+This email has no association with Stoat or Revolt Platforms Ltd.
Learn more about third party instances here:
-https://developers.revolt.chat/faq.html
+https://developers.stoat.chat/faq/
diff --git a/crates/core/database/templates/verify.html b/crates/core/database/templates/verify.html
index 8ab9713d0..10a4ab6ae 100644
--- a/crates/core/database/templates/verify.html
+++ b/crates/core/database/templates/verify.html
@@ -1,7 +1,7 @@
-
+
@@ -42,7 +42,7 @@
|
-
+
|
@@ -104,7 +104,7 @@
This email is intended for {{email}}
- Sent from Revolt
+ Sent from Stoat
Made in Europe
diff --git a/crates/core/database/templates/verify.original.html b/crates/core/database/templates/verify.original.html
index 75ae25cea..107d73077 100644
--- a/crates/core/database/templates/verify.original.html
+++ b/crates/core/database/templates/verify.original.html
@@ -1,35 +1,34 @@
-
-
-
-
-
-
-
-
-
Almost there!
-
To complete your sign up, we just need to verify your email.
-
Confirm
-
-
- This email is intended for {{email}}
- Sent from Revolt
- Made in Europe
-
-
- Revolt Platforms Ltd. is a company incorporated and registered under the
- laws of England and Wales.
- Registered Company Number: 16260658
- Registered Office:
- Suite 5703 Unit 3A, 34-35 Hatton Garden,
- Holborn, United Kingdom, EC1N 8DX
-
+
+
+
+
+
+
+
+
+
+
+
Almost there!
+
To complete your sign up, we just need to verify your email.
+
Confirm
-
-
+
+ This email is intended for {{email}}
+ Sent from Stoat
+ Made in Europe
+
+
+ Revolt Platforms Ltd. is a company incorporated and registered under the
+ laws of England and Wales.
+ Registered Company Number: 16260658
+ Registered Office:
+ Suite 5703 Unit 3A, 34-35 Hatton Garden,
+ Holborn, United Kingdom, EC1N 8DX
+
+
+
+
+
\ No newline at end of file
diff --git a/crates/core/database/templates/verify.txt b/crates/core/database/templates/verify.txt
index 7fa871eef..cafd09d41 100644
--- a/crates/core/database/templates/verify.txt
+++ b/crates/core/database/templates/verify.txt
@@ -4,7 +4,7 @@ To complete your sign up, we just need to verify your email.
Please navigate to: {{url}}
This email is intended for {{email}}
-Sent by Revolt
+Sent by Stoat
Made in Europe
Revolt Platforms Ltd. is a company incorporated and registered under the laws of England and Wales.
diff --git a/crates/core/database/templates/verify.whitelabel.txt b/crates/core/database/templates/verify.whitelabel.txt
index fb54088df..16f1c4b13 100644
--- a/crates/core/database/templates/verify.whitelabel.txt
+++ b/crates/core/database/templates/verify.whitelabel.txt
@@ -5,6 +5,6 @@ Please navigate to: {{url}}
This email is intended for {{email}}
-This email has no association with Revolt or Revolt Platforms Ltd.
+This email has no association with Stoat or Revolt Platforms Ltd.
Learn more about third party instances here:
-https://developers.revolt.chat/faq.html
+https://developers.stoat.chat/faq/
diff --git a/crates/core/files/Cargo.toml b/crates/core/files/Cargo.toml
index 7cc219f1c..c60bf678e 100644
--- a/crates/core/files/Cargo.toml
+++ b/crates/core/files/Cargo.toml
@@ -1,38 +1,48 @@
[package]
name = "revolt-files"
-version = "0.8.8"
+version = "0.13.7"
edition = "2021"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles
"]
description = "Revolt Backend: S3 and encryption subroutines"
+repository = "https://github.com/stoatchat/stoatchat"
[dependencies]
-tracing = "0.1"
+anyhow = { workspace = true }
+thiserror = { workspace = true }
-ffprobe = "0.4.0"
-imagesize = "0.13.0"
-tempfile = "3.12.0"
+tracing = { workspace = true }
-base64 = "0.22.1"
-aes-gcm = "0.10.3"
-typenum = "1.17.0"
+tokio = { workspace = true }
+async-trait = { workspace = true }
-aws-config = "1.5.5"
-aws-sdk-s3 = { version = "1.46.0", features = ["behavior-version-latest"] }
+ffprobe = { workspace = true }
+imagesize = { workspace = true }
+tempfile = { workspace = true }
-revolt-config = { version = "0.8.8", path = "../config", features = [
+base64 = { workspace = true }
+aes-gcm = { workspace = true }
+typenum = { workspace = true }
+
+aws-config = { workspace = true }
+aws-sdk-s3 = { workspace = true, features = ["behavior-version-latest"] }
+
+revolt-config = { workspace = true, features = [
"report-macros",
] }
-revolt-result = { version = "0.8.8", path = "../result" }
+revolt-result = { workspace = true }
# image processing
-jxl-oxide = "0.8.1"
-image = { version = "0.25.2" }
+jxl-oxide = { workspace = true, features = ["image"] }
+image = { workspace = true }
# svg rendering
-usvg = "0.44.0"
-resvg = "0.44.0"
-tiny-skia = "0.11.4"
+usvg = { workspace = true }
+resvg = { workspace = true }
+tiny-skia = { workspace = true }
# encoding
-webp = "0.3.0"
+webp = { workspace = true }
+
+[dev-dependencies]
+uuid = { workspace = true, features = ["v4"] }
diff --git a/crates/core/files/src/implementation/encryption_impl.rs b/crates/core/files/src/implementation/encryption_impl.rs
new file mode 100644
index 000000000..25da9cfd4
--- /dev/null
+++ b/crates/core/files/src/implementation/encryption_impl.rs
@@ -0,0 +1,75 @@
+use aes_gcm::{
+ aead::{Aead, AeadCore, AeadMutInPlace, OsRng},
+ Aes256Gcm, Key, KeyInit, Nonce,
+};
+use base64::{prelude::BASE64_STANDARD, Engine};
+
+use crate::EncryptionRepository;
+
+pub struct EncryptionKey {
+ key: String,
+}
+
+impl EncryptionKey {
+ pub async fn from_config() -> EncryptionKey {
+ EncryptionKey::new(revolt_config::config().await.files.encryption_key)
+ }
+
+ pub fn new(key: String) -> EncryptionKey {
+ EncryptionKey { key }
+ }
+
+ fn create_cipher(&self) -> Aes256Gcm {
+ let key = &BASE64_STANDARD
+ .decode(self.key.clone())
+ .expect("valid base64 string")[..];
+ let key: &Key = key.into();
+ Aes256Gcm::new(key)
+ }
+}
+
+impl EncryptionRepository for EncryptionKey {
+ fn decrypt_buffer(&self, mut buf: Vec, iv: &str) -> anyhow::Result> {
+ let iv = &BASE64_STANDARD.decode(iv).unwrap()[..];
+ let iv: &Nonce = iv.into();
+
+ self.create_cipher()
+ .decrypt_in_place(iv, b"", &mut buf)
+ .map_err(|error| {
+ tracing::error!("{}", error);
+ anyhow::anyhow!("EncryptionRepository: decryption failed")
+ })?;
+
+ Ok(buf)
+ }
+
+ fn encrypt_buffer(&self, buf: &[u8]) -> anyhow::Result<(Vec, String)> {
+ let iv = Aes256Gcm::generate_nonce(&mut OsRng);
+
+ let buf = self.create_cipher().encrypt(&iv, buf).map_err(|error| {
+ tracing::error!("{}", error);
+ anyhow::anyhow!("EncryptionRepository: encryption failed")
+ })?;
+
+ Ok((buf, BASE64_STANDARD.encode(iv)))
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn test_encrypt_and_decrypt() {
+ let encryption =
+ EncryptionKey::new("XkbJ8gBzrouQ+15Ri23xCC81+aZE26Z6+gXzglFxOD4=".to_string());
+
+ let buf: Vec = vec![67];
+ let (ciphertext, iv) = encryption.encrypt_buffer(&buf[..]).unwrap();
+ assert_eq!(ciphertext.len(), 17);
+
+ let plaintext = encryption.decrypt_buffer(ciphertext, &iv).unwrap();
+ assert_eq!(plaintext.len(), 1);
+ assert_eq!(plaintext[0], 67);
+ }
+}
diff --git a/crates/core/files/src/implementation/media_impl.rs b/crates/core/files/src/implementation/media_impl.rs
new file mode 100644
index 000000000..7a7fad848
--- /dev/null
+++ b/crates/core/files/src/implementation/media_impl.rs
@@ -0,0 +1,395 @@
+use anyhow::Result;
+use image::{AnimationDecoder, DynamicImage, ImageBuffer, ImageReader};
+use jxl_oxide::integration::JxlDecoder;
+use revolt_config::report_internal_error;
+use std::io::{BufRead, Read, Seek};
+use tempfile::NamedTempFile;
+use tiny_skia::Pixmap;
+
+use crate::{MediaError, MediaRepository};
+
+pub struct MediaImpl {
+ config: revolt_config::Files,
+}
+
+impl MediaImpl {
+ pub async fn from_config() -> MediaImpl {
+ MediaImpl {
+ config: revolt_config::config().await.files,
+ }
+ }
+
+ pub fn new(config: revolt_config::Files) -> MediaImpl {
+ MediaImpl { config }
+ }
+}
+
+impl MediaRepository for MediaImpl {
+ fn image_size(&self, f: &NamedTempFile) -> Option<(usize, usize)> {
+ if let Ok(size) = imagesize::size(f.path())
+ .inspect_err(|err| tracing::error!("Failed to generate image size! {err:?}"))
+ {
+ Some((size.width, size.height))
+ } else {
+ None
+ }
+ }
+
+ fn is_animated(&self, f: &NamedTempFile, mime: &str) -> Option {
+ match mime {
+ // Current behaviour is to assume GIFs are animated, this checks for at least 2 frames
+ "image/gif" => {
+ let file = std::fs::File::open(f.path()).ok()?;
+ let reader = std::io::BufReader::new(file);
+ let decoder = image::codecs::gif::GifDecoder::new(reader).ok()?;
+ Some(decoder.into_frames().take(2).count() > 1)
+ }
+ "image/png" => {
+ let file = std::fs::File::open(f.path()).ok()?;
+ let reader = std::io::BufReader::new(file);
+ let decoder = image::codecs::png::PngDecoder::new(reader).ok()?;
+ decoder.is_apng().ok()
+ }
+ "image/webp" => {
+ let file = std::fs::File::open(f.path()).ok()?;
+ let reader = std::io::BufReader::new(file);
+ let decoder = image::codecs::webp::WebPDecoder::new(reader).ok()?;
+ Some(decoder.has_animation())
+ }
+ _ => Some(false),
+ }
+ }
+
+ fn image_size_vec(&self, v: &[u8], mime: &str) -> Option<(usize, usize)> {
+ match mime {
+ "image/svg+xml" => {
+ let tree =
+ report_internal_error!(usvg::Tree::from_data(v, &Default::default())).ok()?;
+
+ let size = tree.size();
+ Some((size.width() as usize, size.height() as usize))
+ }
+ _ => {
+ if let Ok(size) = imagesize::blob_size(v)
+ .inspect_err(|err| tracing::error!("Failed to generate image size! {err:?}"))
+ {
+ Some((size.width, size.height))
+ } else {
+ None
+ }
+ }
+ }
+ }
+
+ fn decode_image(
+ &self,
+ reader: &mut R,
+ mime: &str,
+ ) -> Result {
+ match mime {
+ "image/jxl" => {
+ let decoder =
+ JxlDecoder::new(reader).map_err(|e| MediaError::from(anyhow::anyhow!(e)))?;
+
+ DynamicImage::from_decoder(decoder)
+ .map_err(|e| MediaError::from(anyhow::anyhow!(e)))
+ }
+ "image/svg+xml" => {
+ let mut buf = Vec::new();
+ reader
+ .read_to_end(&mut buf)
+ .map_err(|e| MediaError::from(anyhow::anyhow!(e)))?;
+
+ let tree: usvg::Tree = usvg::Tree::from_data(&buf, &Default::default())
+ .map_err(|e| MediaError::from(anyhow::anyhow!(e)))?;
+
+ let size = tree.size();
+ let mut pixmap = Pixmap::new(size.width() as u32, size.height() as u32)
+ .ok_or_else(|| MediaError::ImageProcessingFailed {
+ cause: "failed to create Pixmap, likely zero sized".to_string(),
+ })?;
+
+ let mut pixmap_mut = pixmap.as_mut();
+ resvg::render(&tree, Default::default(), &mut pixmap_mut);
+
+ Ok(DynamicImage::ImageRgba8(
+ ImageBuffer::from_vec(
+ size.width() as u32,
+ size.height() as u32,
+ pixmap.data().to_vec(),
+ )
+ .ok_or_else(|| MediaError::ImageProcessingFailed {
+ cause: "buffer is not big enough".to_string(),
+ })?,
+ ))
+ }
+ _ => {
+ let image: ImageReader<&mut R> = image::ImageReader::new(reader)
+ .with_guessed_format()
+ .map_err(|e| MediaError::from(anyhow::anyhow!(e)))?;
+
+ let image: Result = image
+ .decode()
+ .map_err(|e| MediaError::from(anyhow::anyhow!(e)));
+
+ image
+ }
+ }
+ }
+
+ fn is_valid_image(&self, reader: &mut R, mime: &str) -> bool {
+ match mime {
+ "image/jxl" => jxl_oxide::JxlImage::builder()
+ .read(reader)
+ .inspect_err(|err| tracing::error!("Failed to read JXL! {err:?}"))
+ .is_ok(),
+ _ => !matches!(
+ image::ImageReader::new(reader)
+ .with_guessed_format()
+ .inspect_err(|err| tracing::error!("Failed to read image! {err:?}"))
+ .map(|f| f.decode()),
+ Err(_) | Ok(Err(_))
+ ),
+ }
+ }
+
+ fn create_thumbnail(&self, image: DynamicImage, tag: &str) -> Vec {
+ let [w, h] = self.config.preview.get(tag).unwrap();
+
+ let image = image.thumbnail(image.width().min(*w as u32), image.height().min(*h as u32));
+ let image = match image {
+ DynamicImage::ImageRgb8(_) => image,
+ DynamicImage::ImageRgba8(_) => image,
+ _ => {
+ if image.has_alpha() {
+ image.to_rgba8().into()
+ } else {
+ image.to_rgb8().into()
+ }
+ }
+ };
+
+ let encoder = webp::Encoder::from_image(&image).expect("Could not create encoder.");
+ if self.config.webp_quality != 100.0 {
+ encoder.encode(self.config.webp_quality).to_vec()
+ } else {
+ encoder.encode_lossless().to_vec()
+ }
+ }
+
+ fn video_size(&self, f: &NamedTempFile) -> Option<(i64, i64)> {
+ if let Ok(data) = ffprobe::ffprobe(f.path())
+ .inspect_err(|err| tracing::error!("Failed to ffprobe file! {err:?}"))
+ {
+ for stream in data.streams {
+ if let (Some(w), Some(h)) = (stream.width, stream.height) {
+ return Some((w, h));
+ }
+ }
+
+ None
+ } else {
+ None
+ }
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use crate::{MediaImpl, MediaRepository};
+ use std::io::{Cursor, Write};
+ use tempfile::NamedTempFile;
+
+ #[tokio::test]
+ async fn asset_test_jpeg() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/test.jpeg");
+ assert_eq!(media.image_size_vec(buf, "image/jpeg"), Some((655, 582)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/jpeg").unwrap();
+ media.create_thumbnail(image, "attachments");
+ }
+
+ #[tokio::test]
+ async fn asset_test_jpeg_is_not_animated() {
+ let media = MediaImpl::from_config().await;
+ let mut f = NamedTempFile::new().unwrap();
+ f.write_all(include_bytes!("../../tests/assets/test.jpeg"))
+ .unwrap();
+ assert_eq!(media.is_animated(&f, "image/jpeg"), Some(false));
+ }
+
+ #[tokio::test]
+ async fn asset_test_jpeg_extra_bytes() {
+ let media = MediaImpl::from_config().await;
+ let buf = [
+ &include_bytes!("../../tests/assets/test.jpeg")[..],
+ &[0u8; 16],
+ ]
+ .concat();
+ assert_eq!(media.image_size_vec(&buf, "image/jpeg"), Some((655, 582)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/jpeg").unwrap();
+ media.create_thumbnail(image, "emojis");
+ }
+
+ #[tokio::test]
+ async fn asset_test_png() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/test.png");
+ assert_eq!(media.image_size_vec(buf, "image/png"), Some((900, 900)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/png").unwrap();
+ media.create_thumbnail(image, "emojis");
+ }
+
+ #[tokio::test]
+ async fn asset_test_png_is_not_animated() {
+ let media = MediaImpl::from_config().await;
+ let mut f = NamedTempFile::new().unwrap();
+ f.write_all(include_bytes!("../../tests/assets/test.png"))
+ .unwrap();
+ assert_eq!(media.is_animated(&f, "image/png"), Some(false));
+ }
+
+ #[tokio::test]
+ async fn asset_test_png_extra_bytes() {
+ let media = MediaImpl::from_config().await;
+ let buf = [
+ &include_bytes!("../../tests/assets/test.png")[..],
+ &[0u8; 16],
+ ]
+ .concat();
+ assert_eq!(media.image_size_vec(&buf, "image/png"), Some((900, 900)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/png").unwrap();
+ media.create_thumbnail(image, "emojis");
+ }
+
+ #[tokio::test]
+ async fn asset_test_floating_point_png() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/test-float.png");
+ assert_eq!(media.image_size_vec(buf, "image/png"), Some((300, 300)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/png").unwrap();
+ media.create_thumbnail(image, "avatars");
+ }
+
+ #[tokio::test]
+ async fn asset_test_corrupted_png() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/corrupted.png");
+ assert_eq!(media.image_size_vec(buf, "image/png"), Some((900, 900)));
+
+ let mut reader = Cursor::new(buf);
+ media.decode_image(&mut reader, "image/png").unwrap_err();
+ }
+
+ #[tokio::test]
+ async fn asset_test_animated_png() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/anim-icos.apng");
+ assert_eq!(media.image_size_vec(buf, "image/png"), Some((128, 128)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/png").unwrap();
+ media.create_thumbnail(image, "attachments");
+ }
+
+ #[tokio::test]
+ async fn asset_test_animated_png_is_animated() {
+ let media = MediaImpl::from_config().await;
+ let mut f = NamedTempFile::new().unwrap();
+ f.write_all(include_bytes!("../../tests/assets/anim-icos.apng"))
+ .unwrap();
+ assert_eq!(media.is_animated(&f, "image/png"), Some(true));
+ }
+
+ #[tokio::test]
+ async fn asset_test_jxl() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/dice.jxl");
+ assert_eq!(media.image_size_vec(buf, "image/jxl"), Some((800, 600)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/jxl").unwrap();
+ media.create_thumbnail(image, "attachments");
+ }
+
+ #[tokio::test]
+ async fn asset_test_animated_jxl() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/anim-icos.jxl");
+ assert_eq!(media.image_size_vec(buf, "image/jxl"), Some((128, 128)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/jxl").unwrap();
+ media.create_thumbnail(image, "attachments");
+ }
+
+ #[tokio::test]
+ async fn asset_test_webp() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/dice.webp");
+ assert_eq!(media.image_size_vec(buf, "image/webp"), Some((800, 600)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/webp").unwrap();
+ media.create_thumbnail(image, "attachments");
+ }
+
+ #[tokio::test]
+ async fn asset_test_webp_is_not_animated() {
+ let media = MediaImpl::from_config().await;
+ let mut f = NamedTempFile::new().unwrap();
+ f.write_all(include_bytes!("../../tests/assets/dice.webp"))
+ .unwrap();
+ assert_eq!(media.is_animated(&f, "image/webp"), Some(false));
+ }
+
+ #[tokio::test]
+ async fn asset_test_animated_webp() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/anim-icos.webp");
+ assert_eq!(media.image_size_vec(buf, "image/webp"), Some((128, 128)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/webp").unwrap();
+ media.create_thumbnail(image, "attachments");
+ }
+
+ #[tokio::test]
+ async fn asset_test_animated_webp_is_animated() {
+ let media = MediaImpl::from_config().await;
+ let mut f = NamedTempFile::new().unwrap();
+ f.write_all(include_bytes!("../../tests/assets/anim-icos.webp"))
+ .unwrap();
+ assert_eq!(media.is_animated(&f, "image/webp"), Some(true));
+ }
+
+ #[tokio::test]
+ async fn asset_test_animated_gif() {
+ let media = MediaImpl::from_config().await;
+ let buf = include_bytes!("../../tests/assets/anim-icos.gif");
+ assert_eq!(media.image_size_vec(buf, "image/gif"), Some((128, 128)));
+
+ let mut reader = Cursor::new(buf);
+ let image = media.decode_image(&mut reader, "image/gif").unwrap();
+ media.create_thumbnail(image, "attachments");
+ }
+
+ #[tokio::test]
+ async fn asset_test_animated_gif_is_animated() {
+ let media = MediaImpl::from_config().await;
+ let mut f = NamedTempFile::new().unwrap();
+ f.write_all(include_bytes!("../../tests/assets/anim-icos.gif"))
+ .unwrap();
+ assert_eq!(media.is_animated(&f, "image/gif"), Some(true));
+ }
+}
diff --git a/crates/core/files/src/implementation/mod.rs b/crates/core/files/src/implementation/mod.rs
new file mode 100644
index 000000000..fa96ac22d
--- /dev/null
+++ b/crates/core/files/src/implementation/mod.rs
@@ -0,0 +1,7 @@
+mod encryption_impl;
+mod media_impl;
+mod s3_impl;
+
+pub use encryption_impl::EncryptionKey;
+pub use media_impl::MediaImpl;
+pub use s3_impl::S3Storage;
diff --git a/crates/core/files/src/implementation/s3_impl.rs b/crates/core/files/src/implementation/s3_impl.rs
new file mode 100644
index 000000000..c27d62012
--- /dev/null
+++ b/crates/core/files/src/implementation/s3_impl.rs
@@ -0,0 +1,118 @@
+use std::io::Write;
+
+use anyhow::Context;
+use aws_sdk_s3::{
+ config::{Credentials, Region},
+ Client, Config,
+};
+use revolt_config::FilesS3;
+
+use crate::{EncryptionRepository, FileStorageRepository};
+
+pub struct S3Storage {
+ client: Client,
+ encryption: ER,
+}
+
+impl S3Storage {
+ pub async fn from_config(encryption: ER) -> S3Storage {
+ S3Storage::new(encryption, revolt_config::config().await.files.s3)
+ }
+
+ pub fn new(encryption: ER, s3_config: FilesS3) -> S3Storage {
+ let provider_name = "my-creds";
+ let creds = Credentials::new(
+ s3_config.access_key_id,
+ s3_config.secret_access_key,
+ None,
+ None,
+ provider_name,
+ );
+
+ let config = Config::builder()
+ .region(Region::new(s3_config.region))
+ .endpoint_url(s3_config.endpoint)
+ .force_path_style(s3_config.path_style_buckets)
+ .credentials_provider(creds)
+ .build();
+
+ S3Storage {
+ client: Client::from_conf(config),
+ encryption,
+ }
+ }
+}
+
+#[async_trait::async_trait]
+impl FileStorageRepository for S3Storage {
+ async fn create_bucket(&self, bucket_id: &str) -> anyhow::Result<()> {
+ self.client
+ .create_bucket()
+ .bucket(bucket_id)
+ .send()
+ .await
+ .with_context(|| format!("failed to create bucket {bucket_id}"))?;
+
+ Ok(())
+ }
+
+ async fn fetch_and_decrypt_file(
+ &self,
+ bucket_id: &str,
+ path: &str,
+ iv: &str,
+ ) -> anyhow::Result> {
+ let mut object = self
+ .client
+ .get_object()
+ .bucket(bucket_id)
+ .key(path)
+ .send()
+ .await
+ .with_context(|| format!("failed to get object at {path} in {bucket_id}"))?;
+
+ let mut buf = vec![];
+ while let Some(bytes) = object.body.next().await {
+ let data = bytes?;
+ buf.write_all(&data)?;
+ }
+
+ if iv.is_empty() {
+ Ok(buf)
+ } else {
+ self.encryption.decrypt_buffer(buf, iv)
+ }
+ }
+
+ async fn encrypt_and_upload_file(
+ &self,
+ bucket_id: &str,
+ path: &str,
+ buf: &[u8],
+ ) -> anyhow::Result {
+ let (buf, iv) = self.encryption.encrypt_buffer(buf)?;
+
+ self.client
+ .put_object()
+ .bucket(bucket_id)
+ .key(path)
+ .body(buf.into())
+ .send()
+ .await
+ .with_context(|| format!("failed to put object at {path} in {bucket_id}"))?;
+
+ Ok(iv)
+ }
+
+ async fn delete_file(&self, bucket_id: &str, path: &str) -> anyhow::Result<()> {
+ self.client
+ .delete_object()
+ .bucket(bucket_id)
+ .key(path)
+ .send()
+ .await
+ .with_context(|| format!("failed to delete object at {path} in {bucket_id}"))?;
+
+ Ok(())
+ }
+}
diff --git a/crates/core/files/src/lib.rs b/crates/core/files/src/lib.rs
index dacc988e5..0f71f095c 100644
--- a/crates/core/files/src/lib.rs
+++ b/crates/core/files/src/lib.rs
@@ -1,290 +1,210 @@
-use std::io::{BufRead, Read, Seek, Write};
+mod implementation;
+mod repositories;
-use aes_gcm::{
- aead::{AeadCore, AeadMutInPlace, OsRng},
- Aes256Gcm, Key, KeyInit, Nonce,
-};
-use image::{DynamicImage, ImageBuffer};
-use revolt_config::{config, report_internal_error, FilesS3};
-use revolt_result::{create_error, Result};
+pub use implementation::*;
+pub use repositories::*;
-use aws_sdk_s3::{
- config::{Credentials, Region},
- Client, Config,
-};
+use std::io::{BufRead, Read, Seek};
+
+use image::DynamicImage;
+use revolt_config::{report_internal_error, Files, FilesLimit, FilesS3};
+use revolt_result::Result;
-use base64::prelude::*;
use tempfile::NamedTempFile;
-use tiny_skia::Pixmap;
-/// Size of the authentication tag in the buffer
pub const AUTHENTICATION_TAG_SIZE_BYTES: usize = 16;
-/// Create an S3 client
-pub fn create_client(s3_config: FilesS3) -> Client {
- let provider_name = "my-creds";
- let creds = Credentials::new(
- s3_config.access_key_id,
- s3_config.secret_access_key,
- None,
- None,
- provider_name,
- );
-
- let config = Config::builder()
- .region(Region::new(s3_config.region))
- .endpoint_url(s3_config.endpoint)
- .force_path_style(s3_config.path_style_buckets)
- .credentials_provider(creds)
- .build();
-
- Client::from_conf(config)
-}
-
-/// Create an AES-256-GCM cipher
-pub fn create_cipher(key: &str) -> Aes256Gcm {
- let key = &BASE64_STANDARD.decode(key).expect("valid base64 string")[..];
- let key: &Key = key.into();
- Aes256Gcm::new(key)
-}
-
/// Fetch a file from S3 (and decrypt it)
-pub async fn fetch_from_s3(bucket_id: &str, path: &str, nonce: &str) -> Result> {
- let config = config().await;
- let client = create_client(config.files.s3);
-
- // Send a request for the file
- let mut obj =
- report_internal_error!(client.get_object().bucket(bucket_id).key(path).send().await)?;
-
- // Read the file from remote
- let mut buf = vec![];
- while let Some(bytes) = obj.body.next().await {
- let data = report_internal_error!(bytes)?;
- report_internal_error!(buf.write_all(&data))?;
- // is there a more efficient way to do this?
- // we just want the Vec
- }
-
- // File is not encrypted
- if nonce.is_empty() {
- return Ok(buf);
- }
-
- // Recover nonce as bytes
- let nonce = &BASE64_STANDARD.decode(nonce).unwrap()[..];
- let nonce: &Nonce = nonce.into();
-
- // Decrypt the file
- create_cipher(&config.files.encryption_key)
- .decrypt_in_place(nonce, b"", &mut buf)
- .map_err(|_| create_error!(InternalError))?;
-
- Ok(buf)
+pub async fn fetch_from_s3(bucket_id: &str, path: &str, iv: &str) -> Result> {
+ let encryption = implementation::EncryptionKey::from_config().await;
+ let storage = implementation::S3Storage::from_config(encryption).await;
+ report_internal_error!(storage.fetch_and_decrypt_file(bucket_id, path, iv).await)
}
/// Encrypt and upload a file to S3 (returning its nonce/IV)
pub async fn upload_to_s3(bucket_id: &str, path: &str, buf: &[u8]) -> Result {
- let config = config().await;
- let client = create_client(config.files.s3);
-
- // Generate a nonce
- let nonce = Aes256Gcm::generate_nonce(&mut OsRng);
-
- // Extend the buffer for in-place encryption
- let mut buf = [buf, &[0; AUTHENTICATION_TAG_SIZE_BYTES]].concat();
-
- // Encrypt the file in place
- create_cipher(&config.files.encryption_key)
- .encrypt_in_place(&nonce, b"", &mut buf)
- .map_err(|_| create_error!(InternalError))?;
-
- // Upload the file to remote
- report_internal_error!(
- client
- .put_object()
- .bucket(bucket_id)
- .key(path)
- .body(buf.into())
- .send()
- .await
- )?;
-
- Ok(BASE64_STANDARD.encode(nonce))
+ let encryption = implementation::EncryptionKey::from_config().await;
+ let storage = implementation::S3Storage::from_config(encryption).await;
+ report_internal_error!(storage.encrypt_and_upload_file(bucket_id, path, buf).await)
}
/// Delete a file from S3 by path
pub async fn delete_from_s3(bucket_id: &str, path: &str) -> Result<()> {
- let config = config().await;
- let client = create_client(config.files.s3);
-
- report_internal_error!(
- client
- .delete_object()
- .bucket(bucket_id)
- .key(path)
- .send()
- .await
- )?;
-
- Ok(())
+ let encryption = implementation::EncryptionKey::from_config().await;
+ let storage = implementation::S3Storage::from_config(encryption).await;
+ report_internal_error!(storage.delete_file(bucket_id, path).await)
}
/// Determine size of image at temp file
pub fn image_size(f: &NamedTempFile) -> Option<(usize, usize)> {
- if let Ok(size) = imagesize::size(f.path())
- .inspect_err(|err| tracing::error!("Failed to generate image size! {err:?}"))
- {
- Some((size.width, size.height))
- } else {
- None
- }
+ let media = MediaImpl::new(Files {
+ blocked_mime_types: Default::default(),
+ clamd_host: Default::default(),
+ encryption_key: Default::default(),
+ limit: FilesLimit {
+ max_mega_pixels: 0,
+ max_pixel_side: 0,
+ min_file_size: 0,
+ min_resolution: [0, 0],
+ },
+ preview: Default::default(),
+ s3: FilesS3 {
+ access_key_id: Default::default(),
+ default_bucket: Default::default(),
+ endpoint: Default::default(),
+ path_style_buckets: Default::default(),
+ region: Default::default(),
+ secret_access_key: Default::default(),
+ },
+ scan_mime_types: Default::default(),
+ webp_quality: Default::default(),
+ });
+
+ media.image_size(f)
}
/// Determine size of image with buffer
pub fn image_size_vec(v: &[u8], mime: &str) -> Option<(usize, usize)> {
- match mime {
- "image/svg+xml" => {
- let tree =
- report_internal_error!(usvg::Tree::from_data(v, &Default::default())).ok()?;
+ let media = MediaImpl::new(Files {
+ blocked_mime_types: Default::default(),
+ clamd_host: Default::default(),
+ encryption_key: Default::default(),
+ limit: FilesLimit {
+ max_mega_pixels: 0,
+ max_pixel_side: 0,
+ min_file_size: 0,
+ min_resolution: [0, 0],
+ },
+ preview: Default::default(),
+ s3: FilesS3 {
+ access_key_id: Default::default(),
+ default_bucket: Default::default(),
+ endpoint: Default::default(),
+ path_style_buckets: Default::default(),
+ region: Default::default(),
+ secret_access_key: Default::default(),
+ },
+ scan_mime_types: Default::default(),
+ webp_quality: Default::default(),
+ });
+
+ media.image_size_vec(v, mime)
+}
- let size = tree.size();
- Some((size.width() as usize, size.height() as usize))
- }
- _ => {
- if let Ok(size) = imagesize::blob_size(v)
- .inspect_err(|err| tracing::error!("Failed to generate image size! {err:?}"))
- {
- Some((size.width, size.height))
- } else {
- None
- }
- }
- }
+/// Check whether an image file contains animation data
+pub fn is_animated(f: &NamedTempFile, mime: &str) -> Option {
+ let media = MediaImpl::new(Files {
+ blocked_mime_types: Default::default(),
+ clamd_host: Default::default(),
+ encryption_key: Default::default(),
+ limit: FilesLimit {
+ max_mega_pixels: 0,
+ max_pixel_side: 0,
+ min_file_size: 0,
+ min_resolution: [0, 0],
+ },
+ preview: Default::default(),
+ s3: FilesS3 {
+ access_key_id: Default::default(),
+ default_bucket: Default::default(),
+ endpoint: Default::default(),
+ path_style_buckets: Default::default(),
+ region: Default::default(),
+ secret_access_key: Default::default(),
+ },
+ scan_mime_types: Default::default(),
+ webp_quality: Default::default(),
+ });
+
+ media.is_animated(f, mime)
}
/// Determine size of video at temp file
pub fn video_size(f: &NamedTempFile) -> Option<(i64, i64)> {
- if let Ok(data) = ffprobe::ffprobe(f.path())
- .inspect_err(|err| tracing::error!("Failed to ffprobe file! {err:?}"))
- {
- // Use first valid stream
- for stream in data.streams {
- if let (Some(w), Some(h)) = (stream.width, stream.height) {
- return Some((w, h));
- }
- }
-
- None
- } else {
- None
- }
+ let media = MediaImpl::new(Files {
+ blocked_mime_types: Default::default(),
+ clamd_host: Default::default(),
+ encryption_key: Default::default(),
+ limit: FilesLimit {
+ max_mega_pixels: 0,
+ max_pixel_side: 0,
+ min_file_size: 0,
+ min_resolution: [0, 0],
+ },
+ preview: Default::default(),
+ s3: FilesS3 {
+ access_key_id: Default::default(),
+ default_bucket: Default::default(),
+ endpoint: Default::default(),
+ path_style_buckets: Default::default(),
+ region: Default::default(),
+ secret_access_key: Default::default(),
+ },
+ scan_mime_types: Default::default(),
+ webp_quality: Default::default(),
+ });
+
+ media.video_size(f)
}
/// Decode image from reader
pub fn decode_image(reader: &mut R, mime: &str) -> Result {
- match mime {
- // Read image using jxl-oxide crate
- "image/jxl" => {
- let jxl_image = report_internal_error!(jxl_oxide::JxlImage::builder().read(reader))?;
- if let Ok(frame) = jxl_image.render_frame(0) {
- match frame.color_channels().len() {
- 3 => Ok(DynamicImage::ImageRgb8(
- DynamicImage::ImageRgb32F(
- ImageBuffer::from_vec(
- jxl_image.width(),
- jxl_image.height(),
- frame.image().buf().to_vec(),
- )
- .ok_or_else(|| create_error!(ImageProcessingFailed))?,
- )
- .to_rgb8(),
- )),
- 4 => Ok(DynamicImage::ImageRgba8(
- DynamicImage::ImageRgba32F(
- ImageBuffer::from_vec(
- jxl_image.width(),
- jxl_image.height(),
- frame.image().buf().to_vec(),
- )
- .ok_or_else(|| create_error!(ImageProcessingFailed))?,
- )
- .to_rgba8(),
- )),
- _ => Err(create_error!(ImageProcessingFailed)),
- }
- } else {
- Err(create_error!(ImageProcessingFailed))
- }
- }
- // Read image using resvg
- "image/svg+xml" => {
- // usvg doesn't support Read trait so copy to buffer
- let mut buf = Vec::new();
- report_internal_error!(reader.read_to_end(&mut buf))?;
-
- let tree = report_internal_error!(usvg::Tree::from_data(&buf, &Default::default()))?;
- let size = tree.size();
- let mut pixmap = Pixmap::new(size.width() as u32, size.height() as u32)
- .ok_or_else(|| create_error!(ImageProcessingFailed))?;
-
- let mut pixmap_mut = pixmap.as_mut();
- resvg::render(&tree, Default::default(), &mut pixmap_mut);
-
- Ok(DynamicImage::ImageRgba8(
- ImageBuffer::from_vec(
- size.width() as u32,
- size.height() as u32,
- pixmap.data().to_vec(),
- )
- .ok_or_else(|| create_error!(ImageProcessingFailed))?,
- ))
- }
- // Check if we can read using image-rs crate
- _ => report_internal_error!(report_internal_error!(
- image::ImageReader::new(reader).with_guessed_format()
- )?
- .decode()),
- }
+ let media = MediaImpl::new(Files {
+ blocked_mime_types: Default::default(),
+ clamd_host: Default::default(),
+ encryption_key: Default::default(),
+ limit: FilesLimit {
+ max_mega_pixels: 0,
+ max_pixel_side: 0,
+ min_file_size: 0,
+ min_resolution: [0, 0],
+ },
+ preview: Default::default(),
+ s3: FilesS3 {
+ access_key_id: Default::default(),
+ default_bucket: Default::default(),
+ endpoint: Default::default(),
+ path_style_buckets: Default::default(),
+ region: Default::default(),
+ secret_access_key: Default::default(),
+ },
+ scan_mime_types: Default::default(),
+ webp_quality: Default::default(),
+ });
+
+ report_internal_error!(media.decode_image(reader, mime))
}
/// Check whether given reader has a valid image
pub fn is_valid_image(reader: &mut R, mime: &str) -> bool {
- match mime {
- // Check if we can read using jxl-oxide crate
- "image/jxl" => jxl_oxide::JxlImage::builder()
- .read(reader)
- .inspect_err(|err| tracing::error!("Failed to read JXL! {err:?}"))
- .is_ok(),
- // Check if we can read using image-rs crate
- _ => !matches!(
- image::ImageReader::new(reader)
- .with_guessed_format()
- .inspect_err(|err| tracing::error!("Failed to read image! {err:?}"))
- .map(|f| f.decode()),
- Err(_) | Ok(Err(_))
- ),
- }
+ let media = MediaImpl::new(Files {
+ blocked_mime_types: Default::default(),
+ clamd_host: Default::default(),
+ encryption_key: Default::default(),
+ limit: FilesLimit {
+ max_mega_pixels: 0,
+ max_pixel_side: 0,
+ min_file_size: 0,
+ min_resolution: [0, 0],
+ },
+ preview: Default::default(),
+ s3: FilesS3 {
+ access_key_id: Default::default(),
+ default_bucket: Default::default(),
+ endpoint: Default::default(),
+ path_style_buckets: Default::default(),
+ region: Default::default(),
+ secret_access_key: Default::default(),
+ },
+ scan_mime_types: Default::default(),
+ webp_quality: Default::default(),
+ });
+
+ media.is_valid_image(reader, mime)
}
/// Create thumbnail from given image
pub async fn create_thumbnail(image: DynamicImage, tag: &str) -> Vec {
- // Load configuration
- let config = config().await;
- let [w, h] = config.files.preview.get(tag).unwrap();
-
- // Create thumbnail
- //.resize(width as u32, height as u32, image::imageops::FilterType::Gaussian)
- // resize is about 2.5x slower,
- // thumbnail doesn't have terrible quality
- // so we use thumbnail
- let image = image.thumbnail(image.width().min(*w as u32), image.height().min(*h as u32));
-
- // Encode it into WEBP
- let encoder = webp::Encoder::from_image(&image).expect("Could not create encoder.");
- if config.files.webp_quality != 100.0 {
- encoder.encode(config.files.webp_quality).to_vec()
- } else {
- encoder.encode_lossless().to_vec()
- }
+ let media = MediaImpl::from_config().await;
+ media.create_thumbnail(image, tag)
}
diff --git a/crates/core/files/src/repositories/encryption_repository.rs b/crates/core/files/src/repositories/encryption_repository.rs
new file mode 100644
index 000000000..98e2a4988
--- /dev/null
+++ b/crates/core/files/src/repositories/encryption_repository.rs
@@ -0,0 +1,6 @@
+use anyhow::Result;
+
+pub trait EncryptionRepository: Send + Sync + 'static {
+ fn decrypt_buffer(&self, buf: Vec, iv: &str) -> anyhow::Result>;
+ fn encrypt_buffer(&self, buf: &[u8]) -> Result<(Vec, String)>;
+}
diff --git a/crates/core/files/src/repositories/file_storage_repository.rs b/crates/core/files/src/repositories/file_storage_repository.rs
new file mode 100644
index 000000000..af97be808
--- /dev/null
+++ b/crates/core/files/src/repositories/file_storage_repository.rs
@@ -0,0 +1,22 @@
+use anyhow::Result;
+
+#[async_trait::async_trait]
+pub trait FileStorageRepository: Send + Sync + 'static {
+ async fn create_bucket(&self, bucket_id: &str) -> anyhow::Result<()>;
+
+ async fn fetch_and_decrypt_file(
+ &self,
+ bucket_id: &str,
+ path: &str,
+ iv: &str,
+ ) -> Result>;
+
+ async fn encrypt_and_upload_file(
+ &self,
+ bucket_id: &str,
+ path: &str,
+ buf: &[u8],
+ ) -> Result;
+
+ async fn delete_file(&self, bucket_id: &str, path: &str) -> Result<()>;
+}
diff --git a/crates/core/files/src/repositories/media_repository.rs b/crates/core/files/src/repositories/media_repository.rs
new file mode 100644
index 000000000..6c489d9b3
--- /dev/null
+++ b/crates/core/files/src/repositories/media_repository.rs
@@ -0,0 +1,33 @@
+use anyhow::Result;
+use image::DynamicImage;
+use std::io::{BufRead, Read, Seek};
+use tempfile::NamedTempFile;
+use thiserror::Error;
+
+pub trait MediaRepository: Send + Sync + 'static {
+ fn image_size(&self, f: &NamedTempFile) -> Option<(usize, usize)>;
+
+ fn is_animated(&self, f: &NamedTempFile, mime: &str) -> Option;
+
+ fn image_size_vec(&self, v: &[u8], mime: &str) -> Option<(usize, usize)>;
+
+ fn decode_image(
+ &self,
+ reader: &mut R,
+ mime: &str,
+ ) -> Result;
+
+ fn is_valid_image(&self, reader: &mut R, mime: &str) -> bool;
+
+ fn create_thumbnail(&self, image: DynamicImage, tag: &str) -> Vec;
+
+ fn video_size(&self, f: &NamedTempFile) -> Option<(i64, i64)>;
+}
+
+#[derive(Debug, Error)]
+pub enum MediaError {
+ #[error("image processing failed because {cause}")]
+ ImageProcessingFailed { cause: String },
+ #[error(transparent)]
+ Unknown(#[from] anyhow::Error),
+}
diff --git a/crates/core/files/src/repositories/mod.rs b/crates/core/files/src/repositories/mod.rs
new file mode 100644
index 000000000..cb0749e72
--- /dev/null
+++ b/crates/core/files/src/repositories/mod.rs
@@ -0,0 +1,7 @@
+mod encryption_repository;
+mod file_storage_repository;
+mod media_repository;
+
+pub use encryption_repository::EncryptionRepository;
+pub use file_storage_repository::FileStorageRepository;
+pub use media_repository::{MediaError, MediaRepository};
diff --git a/crates/core/files/tests/assets/anim-icos.apng b/crates/core/files/tests/assets/anim-icos.apng
new file mode 100644
index 000000000..b50cbe364
Binary files /dev/null and b/crates/core/files/tests/assets/anim-icos.apng differ
diff --git a/crates/core/files/tests/assets/anim-icos.gif b/crates/core/files/tests/assets/anim-icos.gif
new file mode 100644
index 000000000..5b81b05eb
Binary files /dev/null and b/crates/core/files/tests/assets/anim-icos.gif differ
diff --git a/crates/core/files/tests/assets/anim-icos.jxl b/crates/core/files/tests/assets/anim-icos.jxl
new file mode 100644
index 000000000..bf3d1e445
Binary files /dev/null and b/crates/core/files/tests/assets/anim-icos.jxl differ
diff --git a/crates/core/files/tests/assets/anim-icos.webp b/crates/core/files/tests/assets/anim-icos.webp
new file mode 100644
index 000000000..ebf34e141
Binary files /dev/null and b/crates/core/files/tests/assets/anim-icos.webp differ
diff --git a/crates/core/files/tests/assets/corrupted.png b/crates/core/files/tests/assets/corrupted.png
new file mode 100644
index 000000000..f0fcce474
Binary files /dev/null and b/crates/core/files/tests/assets/corrupted.png differ
diff --git a/crates/core/files/tests/assets/dice.jxl b/crates/core/files/tests/assets/dice.jxl
new file mode 100644
index 000000000..d1a5a7d0d
Binary files /dev/null and b/crates/core/files/tests/assets/dice.jxl differ
diff --git a/crates/core/files/tests/assets/dice.webp b/crates/core/files/tests/assets/dice.webp
new file mode 100644
index 000000000..ee3dfe535
Binary files /dev/null and b/crates/core/files/tests/assets/dice.webp differ
diff --git a/crates/core/files/tests/assets/test-float.png b/crates/core/files/tests/assets/test-float.png
new file mode 100644
index 000000000..036e9d4d4
Binary files /dev/null and b/crates/core/files/tests/assets/test-float.png differ
diff --git a/crates/core/files/tests/assets/test.jpeg b/crates/core/files/tests/assets/test.jpeg
new file mode 100644
index 000000000..504bd6bcf
Binary files /dev/null and b/crates/core/files/tests/assets/test.jpeg differ
diff --git a/crates/core/files/tests/assets/test.png b/crates/core/files/tests/assets/test.png
new file mode 100644
index 000000000..9de075cdf
Binary files /dev/null and b/crates/core/files/tests/assets/test.png differ
diff --git a/crates/core/files/tests/integration_test.rs b/crates/core/files/tests/integration_test.rs
new file mode 100644
index 000000000..1827a55a8
--- /dev/null
+++ b/crates/core/files/tests/integration_test.rs
@@ -0,0 +1,31 @@
+use std::io::Cursor;
+
+use revolt_files::{EncryptionKey, FileStorageRepository, MediaImpl, MediaRepository, S3Storage};
+
+#[tokio::test]
+async fn test_image_roundtrip_png() {
+ let media = MediaImpl::from_config().await;
+ let encryption = EncryptionKey::from_config().await;
+ let s3 = S3Storage::from_config(encryption).await;
+
+ let buf = include_bytes!("./assets/test.png");
+ let bucket_id = uuid::Uuid::new_v4().to_string();
+
+ s3.create_bucket(&bucket_id).await.unwrap();
+
+ let mut reader = Cursor::new(buf);
+ media.decode_image(&mut reader, "image/png").unwrap();
+
+ let iv = s3
+ .encrypt_and_upload_file(&bucket_id, "/my-file", buf)
+ .await
+ .unwrap();
+
+ let buf = s3
+ .fetch_and_decrypt_file(&bucket_id, "/my-file", &iv)
+ .await
+ .unwrap();
+
+ let mut reader = Cursor::new(buf);
+ media.decode_image(&mut reader, "image/png").unwrap();
+}
diff --git a/crates/core/files/tests/s3_test.rs b/crates/core/files/tests/s3_test.rs
new file mode 100644
index 000000000..95d1afcf1
--- /dev/null
+++ b/crates/core/files/tests/s3_test.rs
@@ -0,0 +1,42 @@
+use revolt_files::{EncryptionKey, FileStorageRepository, S3Storage};
+
+#[tokio::test]
+async fn test_upload_and_download() {
+ let encryption = EncryptionKey::from_config().await;
+ let s3 = S3Storage::from_config(encryption).await;
+
+ let buf = [67];
+ let bucket_id = uuid::Uuid::new_v4().to_string();
+
+ s3.create_bucket(&bucket_id).await.unwrap();
+
+ let iv = s3
+ .encrypt_and_upload_file(&bucket_id, "/my-file", &buf)
+ .await
+ .unwrap();
+
+ let buf = s3
+ .fetch_and_decrypt_file(&bucket_id, "/my-file", &iv)
+ .await
+ .unwrap();
+
+ assert_eq!(buf.len(), 1);
+ assert_eq!(buf[0], 67);
+}
+
+#[tokio::test]
+async fn test_upload_and_delete() {
+ let encryption = EncryptionKey::from_config().await;
+ let s3 = S3Storage::from_config(encryption).await;
+
+ let buf = [67];
+ let bucket_id = uuid::Uuid::new_v4().to_string();
+
+ s3.create_bucket(&bucket_id).await.unwrap();
+
+ s3.encrypt_and_upload_file(&bucket_id, "/my-file", &buf)
+ .await
+ .unwrap();
+
+ s3.delete_file(&bucket_id, "/my-file").await.unwrap();
+}
diff --git a/crates/core/models/Cargo.toml b/crates/core/models/Cargo.toml
index e232fd6cc..c92f3f8a2 100644
--- a/crates/core/models/Cargo.toml
+++ b/crates/core/models/Cargo.toml
@@ -1,10 +1,11 @@
[package]
name = "revolt-models"
-version = "0.8.8"
+version = "0.13.7"
edition = "2021"
license = "MIT"
authors = ["Paul Makles "]
description = "Revolt Backend: API Models"
+repository = "https://github.com/stoatchat/stoatchat"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -20,26 +21,26 @@ default = ["serde", "partials", "rocket"]
[dependencies]
# Core
-revolt-config = { version = "0.8.8", path = "../config" }
-revolt-permissions = { version = "0.8.8", path = "../permissions" }
+revolt-config = { workspace = true }
+revolt-permissions = { workspace = true }
# Utility
-regex = "1.11"
-indexmap = "1.9.3"
-once_cell = "1.17.1"
-num_enum = "0.6.1"
+regex = { workspace = true }
+indexmap = { workspace = true }
+once_cell = { workspace = true }
+num_enum = { workspace = true }
# Rocket
-rocket = { optional = true, version = "0.5.0-rc.2", default-features = false }
+rocket = { workspace = true, optional = true }
# Serialisation
-revolt_optional_struct = { version = "0.2.0", optional = true }
-serde = { version = "1", features = ["derive"], optional = true }
-iso8601-timestamp = { version = "0.2.11", features = ["schema", "bson"] }
+revolt_optional_struct = { workspace = true, optional = true }
+serde = { workspace = true, optional = true }
+iso8601-timestamp = { workspace = true, features = ["schema", "bson"] }
# Spec Generation
-schemars = { version = "0.8.8", optional = true, features = ["indexmap1"] }
-utoipa = { version = "4.2.3", optional = true }
+schemars = { workspace = true, features = ["indexmap2"], optional = true }
+utoipa = { workspace = true, optional = true }
# Validation
-validator = { version = "0.16.0", optional = true, features = ["derive"] }
+validator = { workspace = true, features = ["derive"], optional = true }
diff --git a/crates/core/models/src/v0/channels.rs b/crates/core/models/src/v0/channels.rs
index ec25257b8..faddee749 100644
--- a/crates/core/models/src/v0/channels.rs
+++ b/crates/core/models/src/v0/channels.rs
@@ -1,4 +1,5 @@
-use super::File;
+#![allow(deprecated)]
+use super::{File, UserVoiceState};
use revolt_permissions::{Override, OverrideField};
use std::collections::{HashMap, HashSet};
@@ -107,46 +108,27 @@ auto_derived!(
serde(skip_serializing_if = "crate::if_false", default)
)]
nsfw: bool,
- },
- /// Voice channel belonging to a server
- VoiceChannel {
- /// Unique Id
- #[cfg_attr(feature = "serde", serde(rename = "_id"))]
- id: String,
- /// Id of the server this channel belongs to
- server: String,
-
- /// Display name of the channel
- name: String,
- #[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
- /// Channel description
- description: Option,
- /// Custom icon attachment
- #[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
- icon: Option,
- /// Default permissions assigned to users in this channel
+ /// Voice Information for when this channel is also a voice channel
#[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
- default_permissions: Option,
- /// Permissions assigned based on role to this channel
- #[cfg_attr(
- feature = "serde",
- serde(
- default = "HashMap::::new",
- skip_serializing_if = "HashMap::::is_empty"
- )
- )]
- role_permissions: HashMap,
+ voice: Option,
- /// Whether this channel is marked as not safe for work
- #[cfg_attr(
- feature = "serde",
- serde(skip_serializing_if = "crate::if_false", default)
- )]
- nsfw: bool,
+ /// The channel's slowmode delay in seconds
+ #[serde(skip_serializing_if = "Option::is_none")]
+ slowmode: Option,
},
}
+ /// Voice information for a channel
+ #[derive(Default)]
+ #[cfg_attr(feature = "validator", derive(validator::Validate))]
+ pub struct VoiceInformation {
+ /// Maximium amount of users allowed in the voice channel at once
+ #[cfg_attr(feature = "validator", validate(range(min = 1)))]
+ #[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
+ pub max_users: Option,
+ }
+
/// Partial representation of a channel
#[derive(Default)]
pub struct PartialChannel {
@@ -170,6 +152,10 @@ auto_derived!(
pub default_permissions: Option,
#[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
pub last_message_id: Option,
+ #[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
+ pub voice: Option,
+ #[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
+ pub slowmode: Option,
}
/// Optional fields on channel object
@@ -177,6 +163,7 @@ auto_derived!(
Description,
Icon,
DefaultPermissions,
+ Voice,
}
/// New webhook information
@@ -205,6 +192,13 @@ auto_derived!(
/// Whether this channel is archived
pub archived: Option,
+ /// Voice Information for voice channels
+ pub voice: Option,
+
+ /// The channel's slow mode delay in seconds, up to 6 hours
+ #[cfg_attr(feature = "validator", validate(range(min = 0, max = 21600)))]
+ pub slowmode: Option,
+
/// Fields to remove from channel
#[cfg_attr(feature = "serde", serde(default))]
pub remove: Vec,
@@ -260,6 +254,10 @@ auto_derived!(
/// Whether this channel is age restricted
#[serde(skip_serializing_if = "Option::is_none")]
pub nsfw: Option,
+
+ /// Voice Information for when this channel is also a voice channel
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub voice: Option,
}
/// New default permissions
@@ -270,7 +268,7 @@ auto_derived!(
permissions: u64,
},
Field {
- /// Allow / deny values to set for members in this `TextChannel` or `VoiceChannel`
+ /// Allow / deny values to set for members in this server channel
permissions: Override,
},
}
@@ -289,9 +287,38 @@ auto_derived!(
}
/// Voice server token response
- pub struct LegacyCreateVoiceUserResponse {
+ pub struct CreateVoiceUserResponse {
/// Token for authenticating with the voice server
- token: String,
+ pub token: String,
+ /// Url of the livekit server to connect to
+ pub url: String,
+ }
+
+ /// Voice state for a channel
+ pub struct ChannelVoiceState {
+ pub id: String,
+ /// The states of the users who are connected to the channel
+ pub participants: Vec,
+ }
+
+ /// Join a voice channel
+ pub struct DataJoinCall {
+ /// Name of the node to join
+ pub node: Option,
+ /// Whether to force disconnect any other existing voice connections
+ ///
+ /// Useful for disconnecting on another device and joining on a new.
+ pub force_disconnect: Option,
+ /// Users which should be notified of the call starting
+ ///
+ /// Only used when the user is the first one connected.
+ pub recipients: Option>,
+ }
+
+ pub struct ChannelSlowmode {
+ pub channel_id: String,
+ pub duration: u64,
+ pub retry_after: u64,
}
);
@@ -302,8 +329,7 @@ impl Channel {
Channel::DirectMessage { id, .. }
| Channel::Group { id, .. }
| Channel::SavedMessages { id, .. }
- | Channel::TextChannel { id, .. }
- | Channel::VoiceChannel { id, .. } => id,
+ | Channel::TextChannel { id, .. } => id,
}
}
@@ -315,9 +341,7 @@ impl Channel {
match self {
Channel::DirectMessage { .. } => None,
Channel::SavedMessages { .. } => Some("Saved Messages"),
- Channel::TextChannel { name, .. }
- | Channel::Group { name, .. }
- | Channel::VoiceChannel { name, .. } => Some(name),
+ Channel::TextChannel { name, .. } | Channel::Group { name, .. } => Some(name),
}
}
}
diff --git a/crates/core/models/src/v0/emojis.rs b/crates/core/models/src/v0/emojis.rs
index 2d7c015e1..58d47429a 100644
--- a/crates/core/models/src/v0/emojis.rs
+++ b/crates/core/models/src/v0/emojis.rs
@@ -54,4 +54,22 @@ auto_derived!(
#[serde(default)]
pub nsfw: bool,
}
+
+ /// Partial emoji representation
+ #[derive(Default)]
+ pub struct PartialEmoji {
+ #[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
+ pub name: Option,
+ }
+
+ /// Edit emoji information
+ #[cfg_attr(feature = "validator", derive(Validate))]
+ pub struct DataEditEmoji {
+ /// Emoji name
+ #[cfg_attr(
+ feature = "validator",
+ validate(length(min = 1, max = 32), regex = "RE_EMOJI")
+ )]
+ pub name: Option,
+ }
);
diff --git a/crates/core/models/src/v0/files.rs b/crates/core/models/src/v0/files.rs
index b996483cb..2d68b7f2b 100644
--- a/crates/core/models/src/v0/files.rs
+++ b/crates/core/models/src/v0/files.rs
@@ -46,8 +46,13 @@ auto_derived!(
File,
/// File contains textual data and should be displayed as such
Text,
- /// File is an image with specific dimensions
- Image { width: usize, height: usize },
+ /// File is an image with specific dimensions, and may be animated
+ Image {
+ width: usize,
+ height: usize,
+ thumbhash: Option>,
+ animated: Option,
+ },
/// File is a video with specific dimensions
Video { width: usize, height: usize },
/// File is audio
diff --git a/crates/core/models/src/v0/messages.rs b/crates/core/models/src/v0/messages.rs
index 2116d98ba..cb8f3f246 100644
--- a/crates/core/models/src/v0/messages.rs
+++ b/crates/core/models/src/v0/messages.rs
@@ -132,6 +132,11 @@ auto_derived!(
MessagePinned { id: String, by: String },
#[serde(rename = "message_unpinned")]
MessageUnpinned { id: String, by: String },
+ #[serde(rename = "call_started")]
+ CallStarted {
+ by: String,
+ finished_at: Option,
+ },
}
/// Name and / or avatar override information
@@ -199,6 +204,9 @@ auto_derived!(
pub image: Option,
/// Message content or system message information
pub body: String,
+ /// The raw body, if the body has been rendered
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub raw_body: Option,
/// Unique tag, usually the channel ID
pub tag: String,
/// Timestamp at which this notification was created
@@ -253,7 +261,7 @@ auto_derived!(
pub nonce: Option,
/// Message content to send
- #[cfg_attr(feature = "validator", validate(length(min = 0, max = 2000)))]
+ #[cfg_attr(feature = "validator", validate(length(min = 0)))]
pub content: Option,
/// Attachments to include in message
pub attachments: Option>,
@@ -337,7 +345,7 @@ auto_derived!(
#[cfg_attr(feature = "validator", derive(Validate))]
pub struct DataEditMessage {
/// New message content
- #[cfg_attr(feature = "validator", validate(length(min = 1, max = 2000)))]
+ #[cfg_attr(feature = "validator", validate(length(min = 1)))]
pub content: Option,
/// Embeds to include in the message
#[cfg_attr(feature = "validator", validate(length(min = 0, max = 10)))]
@@ -383,6 +391,7 @@ auto_derived!(
);
/// Message Author Abstraction
+#[derive(Clone)]
pub enum MessageAuthor<'a> {
User(&'a User),
Webhook(&'a Webhook),
@@ -445,6 +454,7 @@ impl From for String {
}
SystemMessage::MessagePinned { .. } => "Message pinned.".to_string(),
SystemMessage::MessageUnpinned { .. } => "Message unpinned.".to_string(),
+ SystemMessage::CallStarted { .. } => "Call started.".to_string(),
}
}
}
@@ -505,6 +515,7 @@ impl PushNotification {
icon,
image,
body,
+ raw_body: None,
tag: channel.id().to_string(),
timestamp,
url: format!("{}/channel/{}/{}", config.hosts.app, channel.id(), msg.id),
diff --git a/crates/core/models/src/v0/server_bans.rs b/crates/core/models/src/v0/server_bans.rs
index b6debfcdc..940a3738b 100644
--- a/crates/core/models/src/v0/server_bans.rs
+++ b/crates/core/models/src/v0/server_bans.rs
@@ -19,6 +19,9 @@ auto_derived!(
/// Ban reason
#[cfg_attr(feature = "validator", validate(length(min = 0, max = 1024)))]
pub reason: Option,
+ /// Messages to delete in seconds
+ #[cfg_attr(feature = "validator", validate(range(min = 0, max = 604800)))]
+ pub delete_message_seconds: Option,
}
/// Just enough information to list a ban
diff --git a/crates/core/models/src/v0/server_members.rs b/crates/core/models/src/v0/server_members.rs
index 048f696bc..41ec99ae8 100644
--- a/crates/core/models/src/v0/server_members.rs
+++ b/crates/core/models/src/v0/server_members.rs
@@ -31,6 +31,14 @@ pub static RE_COLOUR: Lazy = Lazy::new(|| {
Regex::new(r"(?i)^(?:[a-z ]+|var\(--[a-z\d-]+\)|rgba?\([\d, ]+\)|#[a-f0-9]+|(repeating-)?(linear|conic|radial)-gradient\(([a-z ]+|var\(--[a-z\d-]+\)|rgba?\([\d, ]+\)|#[a-f0-9]+|\d+deg)([ ]+(\d{1,3}%|0))?(,[ ]*([a-z ]+|var\(--[a-z\d-]+\)|rgba?\([\d, ]+\)|#[a-f0-9]+)([ ]+(\d{1,3}%|0))?)+\))$").unwrap()
});
+fn default_true() -> bool {
+ true
+}
+
+fn is_true(x: &bool) -> bool {
+ *x
+}
+
auto_derived_partial!(
/// Server Member
pub struct Member {
@@ -57,6 +65,13 @@ auto_derived_partial!(
/// Timestamp this member is timed out until
#[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
pub timeout: Option,
+
+ /// Whether the member is server-wide voice muted
+ #[serde(skip_serializing_if = "is_true", default = "default_true")]
+ pub can_publish: bool,
+ /// Whether the member is server-wide voice deafened
+ #[serde(skip_serializing_if = "is_true", default = "default_true")]
+ pub can_receive: bool,
},
"PartialMember"
);
@@ -77,7 +92,10 @@ auto_derived!(
Avatar,
Roles,
Timeout,
+ CanReceive,
+ CanPublish,
JoinedAt,
+ VoiceChannel,
}
/// Member removal intention
@@ -124,6 +142,12 @@ auto_derived!(
pub roles: Option>,
/// Timestamp this member is timed out until
pub timeout: Option,
+ /// server-wide voice muted
+ pub can_publish: Option,
+ /// server-wide voice deafened
+ pub can_receive: Option,
+ /// voice channel to move to if already in a voice channel
+ pub voice_channel: Option,
/// Fields to remove from channel object
#[cfg_attr(feature = "serde", serde(default))]
pub remove: Vec,
diff --git a/crates/core/models/src/v0/servers.rs b/crates/core/models/src/v0/servers.rs
index c3ed20a8b..2aab765e6 100644
--- a/crates/core/models/src/v0/servers.rs
+++ b/crates/core/models/src/v0/servers.rs
@@ -85,6 +85,9 @@ auto_derived_partial!(
auto_derived_partial!(
/// Role
pub struct Role {
+ /// Unique Id
+ #[cfg_attr(feature = "serde", serde(rename = "_id"))]
+ pub id: String,
/// Role name
pub name: String,
/// Permissions available to this role
@@ -103,6 +106,9 @@ auto_derived_partial!(
/// Ranking of this role
#[cfg_attr(feature = "serde", serde(default))]
pub rank: i64,
+ /// Role icon
+ #[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
+ pub icon: Option,
},
"PartialRole"
);
@@ -120,6 +126,7 @@ auto_derived!(
/// Optional fields on server object
pub enum FieldsRole {
Colour,
+ Icon,
}
/// Channel category
@@ -181,6 +188,7 @@ auto_derived!(
}
/// Response after creating new role
+ // TODO: remove this in favor of just Role
pub struct NewRoleResponse {
/// Id of the role
pub id: String,
@@ -248,6 +256,9 @@ auto_derived!(
/// Must be enabled in order to show up on [Revolt Discover](https://rvlt.gg).
pub analytics: Option,
+ /// User id of the new owner
+ pub owner: Option,
+
/// Fields to remove from server object
#[cfg_attr(feature = "serde", serde(default))]
pub remove: Vec,
@@ -271,6 +282,11 @@ auto_derived!(
///
/// **Removed** - no effect, use the edit server role positions route
pub rank: Option,
+ /// Role icon
+ ///
+ /// Provide an Autumn attachment Id.
+ #[cfg_attr(feature = "validator", validate(length(min = 1, max = 128)))]
+ pub icon: Option,
/// Fields to remove from role object
#[cfg_attr(feature = "serde", serde(default))]
pub remove: Vec,
diff --git a/crates/core/models/src/v0/users.rs b/crates/core/models/src/v0/users.rs
index fc681e003..73957768b 100644
--- a/crates/core/models/src/v0/users.rs
+++ b/crates/core/models/src/v0/users.rs
@@ -1,3 +1,4 @@
+use iso8601_timestamp::Timestamp;
use once_cell::sync::Lazy;
use regex::Regex;
@@ -279,6 +280,19 @@ auto_derived!(
}
);
+auto_derived_partial!(
+ /// Voice State information for a user
+ pub struct UserVoiceState {
+ pub id: String,
+ pub joined_at: Timestamp,
+ pub is_receiving: bool,
+ pub is_publishing: bool,
+ pub screensharing: bool,
+ pub camera: bool,
+ },
+ "PartialUserVoiceState"
+);
+
pub trait CheckRelationship {
fn with(&self, user: &str) -> RelationshipStatus;
}
diff --git a/crates/core/parser/Cargo.toml b/crates/core/parser/Cargo.toml
index a6e1869b7..5bb079acf 100644
--- a/crates/core/parser/Cargo.toml
+++ b/crates/core/parser/Cargo.toml
@@ -1,10 +1,11 @@
[package]
name = "revolt-parser"
-version = "0.8.8"
+version = "0.13.7"
edition = "2021"
license = "MIT"
authors = ["Zomatree ", "Paul Makles "]
description = "Revolt Backend: Message Parser"
+repository = "https://github.com/stoatchat/stoatchat"
[dependencies]
-logos = { version = "0.15" }
+logos = { workspace = true }
diff --git a/crates/core/parser/src/lib.rs b/crates/core/parser/src/lib.rs
index 4ebbac630..27211c1ba 100644
--- a/crates/core/parser/src/lib.rs
+++ b/crates/core/parser/src/lib.rs
@@ -16,23 +16,29 @@ pub enum MessageToken<'a> {
UserMention(&'a str),
#[regex("<%(?&id)>", |lex| &lex.slice()[2..lex.slice().len() - 1],)]
RoleMention(&'a str),
+ #[regex("<#(?&id)>", |lex| &lex.slice()[2..lex.slice().len() - 1],)]
+ ChannelMention(&'a str),
+ #[regex(":(?&id):", |lex| &lex.slice()[1..lex.slice().len() - 1],)]
+ Emoji(&'a str),
#[token("@everyone")]
MentionEveryone,
#[token("@online")]
- MentionOnline
+ MentionOnline,
}
#[derive(Debug, Clone, PartialEq, Default)]
pub struct MessageResults {
pub user_mentions: HashSet,
pub role_mentions: HashSet,
+ pub channel_mentions: HashSet,
+ pub emojis: HashSet,
pub mentions_everyone: bool,
- pub mentions_online: bool
+ pub mentions_online: bool,
}
struct MessageParserIterator<'a, I> {
inner: I,
- temp: VecDeque>
+ temp: VecDeque>,
}
impl<'a, I: Iterator- >> Iterator for MessageParserIterator<'a, I> {
@@ -55,11 +61,11 @@ impl<'a, I: Iterator
- >> Iterator for MessageParserIterato
if next_token == Some(MessageToken::CodeblockMarker(ty)) {
self.temp.clear();
self.temp.push_back(MessageToken::CodeblockMarker(ty));
- break next_token
+ break next_token;
} else if let Some(token) = next_token {
self.temp.push_back(token);
} else {
- break Some(MessageToken::CodeblockMarker(ty))
+ break Some(MessageToken::CodeblockMarker(ty));
}
}
} else {
@@ -69,10 +75,10 @@ impl<'a, I: Iterator
- >> Iterator for MessageParserIterato
}
}
-pub fn parse_message_iter(text: &str) -> impl Iterator
- + '_ {
+pub fn parse_message_iter(text: &str) -> impl Iterator
- > + '_ {
MessageParserIterator {
inner: MessageToken::lexer(text).flatten(),
- temp: VecDeque::new()
+ temp: VecDeque::new(),
}
}
@@ -82,13 +88,23 @@ pub fn parse_message(text: &str) -> MessageResults {
for token in parse_message_iter(text) {
match token {
MessageToken::Escape => {}
- MessageToken::CodeblockMarker(_) => {},
- MessageToken::UserMention(id) => { results.user_mentions.insert(id.to_string()); },
- MessageToken::RoleMention(id) => { results.role_mentions.insert(id.to_string()); },
+ MessageToken::CodeblockMarker(_) => {}
+ MessageToken::UserMention(id) => {
+ results.user_mentions.insert(id.to_string());
+ }
+ MessageToken::RoleMention(id) => {
+ results.role_mentions.insert(id.to_string());
+ }
+ MessageToken::ChannelMention(id) => {
+ results.channel_mentions.insert(id.to_string());
+ }
+ MessageToken::Emoji(id) => {
+ results.emojis.insert(id.to_string());
+ }
MessageToken::MentionEveryone => results.mentions_everyone = true,
MessageToken::MentionOnline => results.mentions_online = true,
};
- };
+ }
results
}
@@ -109,7 +125,10 @@ mod tests {
let output = parse_message_iter("Hello <@01FD58YK5W7QRV5H3D64KTQYX3>.").collect::>();
assert_eq!(output.len(), 1);
- assert_eq!(output[0], MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3"));
+ assert_eq!(
+ output[0],
+ MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
}
#[test]
@@ -117,7 +136,10 @@ mod tests {
let output = parse_message_iter("Hello <%01FD58YK5W7QRV5H3D64KTQYX3>.").collect::>();
assert_eq!(output.len(), 1);
- assert_eq!(output[0], MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3"));
+ assert_eq!(
+ output[0],
+ MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
}
#[test]
@@ -138,29 +160,57 @@ mod tests {
#[test]
fn test_everything() {
- let output = parse_message_iter("Hello <@01FD58YK5W7QRV5H3D64KTQYX3>, <%01FD58YK5W7QRV5H3D64KTQYX3>, @everyone and @online.").collect::>();
-
- assert_eq!(output.len(), 4);
- assert_eq!(output[0], MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[1], MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[2], MessageToken::MentionEveryone);
- assert_eq!(output[3], MessageToken::MentionOnline);
+ let output = parse_message_iter("Hello <@01FD58YK5W7QRV5H3D64KTQYX3>, <%01FD58YK5W7QRV5H3D64KTQYX3>, <#01FD58YK5W7QRV5H3D64KTQYX3> @everyone and @online. :01FD58YK5W7QRV5H3D64KTQYX3:").collect::>();
+
+ assert_eq!(output.len(), 6);
+ assert_eq!(
+ output[0],
+ MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[1],
+ MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[2],
+ MessageToken::ChannelMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(output[3], MessageToken::MentionEveryone);
+ assert_eq!(output[4], MessageToken::MentionOnline);
+ assert_eq!(output[5], MessageToken::Emoji("01FD58YK5W7QRV5H3D64KTQYX3"));
}
#[test]
fn test_everything_no_spaces() {
- let output = parse_message_iter("<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online").collect::>();
-
- assert_eq!(output.len(), 4);
- assert_eq!(output[0], MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[1], MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[2], MessageToken::MentionEveryone);
- assert_eq!(output[3], MessageToken::MentionOnline);
+ let output = parse_message_iter(
+ "<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3><#01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online:01FD58YK5W7QRV5H3D64KTQYX3:",
+ )
+ .collect::>();
+
+ assert_eq!(output.len(), 6);
+ assert_eq!(
+ output[0],
+ MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[1],
+ MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[2],
+ MessageToken::ChannelMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(output[3], MessageToken::MentionEveryone);
+ assert_eq!(output[4], MessageToken::MentionOnline);
+ assert_eq!(output[5], MessageToken::Emoji("01FD58YK5W7QRV5H3D64KTQYX3"));
}
#[test]
fn test_codeblock_no_mentions() {
- let output = parse_message_iter("```\n<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online\n```").collect::>();
+ let output = parse_message_iter(
+ "```\n<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3><#01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online:01FD58YK5W7QRV5H3D64KTQYX3:\n```",
+ )
+ .collect::>();
assert_eq!(output.len(), 2);
assert_eq!(output[0], MessageToken::CodeblockMarker(3));
@@ -169,19 +219,36 @@ mod tests {
#[test]
fn test_uncontained_codeblock_should_mention() {
- let output = parse_message_iter("```\n<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online").collect::>();
+ let output = parse_message_iter(
+ "```\n<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3><#01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online:01FD58YK5W7QRV5H3D64KTQYX3:",
+ )
+ .collect::>();
- assert_eq!(output.len(), 5);
+ assert_eq!(output.len(), 7);
assert_eq!(output[0], MessageToken::CodeblockMarker(3));
- assert_eq!(output[1], MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[2], MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[3], MessageToken::MentionEveryone);
- assert_eq!(output[4], MessageToken::MentionOnline);
+ assert_eq!(
+ output[1],
+ MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[2],
+ MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[3],
+ MessageToken::ChannelMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(output[4], MessageToken::MentionEveryone);
+ assert_eq!(output[5], MessageToken::MentionOnline);
+ assert_eq!(output[6], MessageToken::Emoji("01FD58YK5W7QRV5H3D64KTQYX3"));
}
#[test]
fn test_inline_codeblock_no_mentions() {
- let output = parse_message_iter("`<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online`").collect::>();
+ let output = parse_message_iter(
+ "`<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3><#01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online:01FD58YK5W7QRV5H3D64KTQYX3:`",
+ )
+ .collect::>();
assert_eq!(output.len(), 2);
assert_eq!(output[0], MessageToken::CodeblockMarker(1));
@@ -190,19 +257,33 @@ mod tests {
#[test]
fn test_uncontained_inline_codeblock_should_mention() {
- let output = parse_message_iter("`<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online").collect::>();
+ let output = parse_message_iter(
+ "`<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3><#01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online:01FD58YK5W7QRV5H3D64KTQYX3:",
+ )
+ .collect::>();
- assert_eq!(output.len(), 5);
+ assert_eq!(output.len(), 7);
assert_eq!(output[0], MessageToken::CodeblockMarker(1));
- assert_eq!(output[1], MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[2], MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3"));
- assert_eq!(output[3], MessageToken::MentionEveryone);
- assert_eq!(output[4], MessageToken::MentionOnline);
+ assert_eq!(
+ output[1],
+ MessageToken::UserMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[2],
+ MessageToken::RoleMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(
+ output[3],
+ MessageToken::ChannelMention("01FD58YK5W7QRV5H3D64KTQYX3")
+ );
+ assert_eq!(output[4], MessageToken::MentionEveryone);
+ assert_eq!(output[5], MessageToken::MentionOnline);
+ assert_eq!(output[6], MessageToken::Emoji("01FD58YK5W7QRV5H3D64KTQYX3"));
}
#[test]
fn test_codeblock_with_language_no_mentions() {
- let output = parse_message_iter("```rust\n<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online```").collect::>();
+ let output = parse_message_iter("```rust\n<@01FD58YK5W7QRV5H3D64KTQYX3><%01FD58YK5W7QRV5H3D64KTQYX3><#01FD58YK5W7QRV5H3D64KTQYX3>@everyone@online:01FD58YK5W7QRV5H3D64KTQYX3:```").collect::>();
assert_eq!(output.len(), 2);
assert_eq!(output[0], MessageToken::CodeblockMarker(3));
@@ -220,7 +301,8 @@ mod tests {
#[test]
fn test_double_inline_codeblock_with_backticks_inside() {
- let output = parse_message_iter("``this `should` not `ping` @everyone``").collect::>();
+ let output =
+ parse_message_iter("``this `should` not `ping` @everyone``").collect::>();
assert_eq!(output.len(), 2);
assert_eq!(output[0], MessageToken::CodeblockMarker(2));
@@ -238,7 +320,8 @@ mod tests {
#[test]
fn test_escaped_codeblock() {
- let output = parse_message_iter("i am ~~not~~ pinging \\`@everyone` ok.").collect::>();
+ let output =
+ parse_message_iter("i am ~~not~~ pinging \\`@everyone` ok.").collect::>();
assert_eq!(output.len(), 3);
assert_eq!(output[0], MessageToken::Escape);
@@ -253,4 +336,4 @@ mod tests {
assert_eq!(output.len(), 1);
assert_eq!(output[0], MessageToken::Escape);
}
-}
\ No newline at end of file
+}
diff --git a/crates/core/permissions/Cargo.toml b/crates/core/permissions/Cargo.toml
index d5ee32165..03ed629d6 100644
--- a/crates/core/permissions/Cargo.toml
+++ b/crates/core/permissions/Cargo.toml
@@ -1,10 +1,11 @@
[package]
name = "revolt-permissions"
-version = "0.8.8"
+version = "0.13.7"
edition = "2021"
license = "MIT"
authors = ["Paul Makles "]
description = "Revolt Backend: Permission Logic"
+repository = "https://github.com/stoatchat/stoatchat"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -17,23 +18,23 @@ try-from-primitive = ["dep:num_enum"]
[dev-dependencies]
# Async
-async-std = { version = "1.8.0", features = ["attributes"] }
+async-std = { workspace = true, features = ["attributes"] }
[dependencies]
# Core
-revolt-result = { version = "0.8.8", path = "../result" }
+revolt-result = { workspace = true }
# Utility
-auto_ops = "0.3.0"
-once_cell = "1.17"
-num_enum = { version = "0.6.1", optional = true }
+auto_ops = { workspace = true }
+once_cell = { workspace = true }
+num_enum = { workspace = true, optional = true }
# Async
-async-trait = "0.1.51"
+async-trait = { workspace = true }
# Serialisation
-serde = { version = "1", features = ["derive"], optional = true }
-bson = { version = "2.1.0", optional = true }
+serde = { workspace = true, optional = true }
+bson = { workspace = true, optional = true }
# Spec Generation
-schemars = { version = "0.8.8", optional = true }
+schemars = { workspace = true, optional = true }
diff --git a/crates/core/permissions/src/impl.rs b/crates/core/permissions/src/impl.rs
index 9975eeda5..064fee3e6 100644
--- a/crates/core/permissions/src/impl.rs
+++ b/crates/core/permissions/src/impl.rs
@@ -61,6 +61,15 @@ pub async fn calculate_server_permissions(query: &mut P) ->
permissions.apply(role_override);
}
+ if !query.do_we_have_publish_overwrites().await {
+ permissions.revoke(ChannelPermission::Speak as u64);
+ permissions.revoke(ChannelPermission::Video as u64);
+ }
+
+ if !query.do_we_have_receive_overwrites().await {
+ permissions.revoke(ChannelPermission::Listen as u64);
+ }
+
if query.are_we_timed_out().await {
permissions.restrict(*ALLOW_IN_TIMEOUT);
}
diff --git a/crates/core/permissions/src/models/channel.rs b/crates/core/permissions/src/models/channel.rs
index aca22b5c2..fc6fe49a0 100644
--- a/crates/core/permissions/src/models/channel.rs
+++ b/crates/core/permissions/src/models/channel.rs
@@ -75,6 +75,8 @@ pub enum ChannelPermission {
Masquerade = 1 << 28,
/// React to messages with emojis
React = 1 << 29,
+ /// Bypass slowmode
+ BypassSlowmode = 1 << 39,
// * Voice permissions
/// Connect to a voice channel
@@ -89,6 +91,8 @@ pub enum ChannelPermission {
DeafenMembers = 1 << 34,
/// Move members between voice channels
MoveMembers = 1 << 35,
+ /// Listen to other users
+ Listen = 1 << 36,
// * Channel permissions two electric boogaloo
/// Mention everyone and online members
@@ -97,7 +101,7 @@ pub enum ChannelPermission {
MentionRoles = 1 << 38,
// * Misc. permissions
- // % Bits 38 to 52: free area
+ // % Bits 39 to 52: free area
// % Bits 53 to 64: do not use
// * Grant all permissions
@@ -130,14 +134,16 @@ pub static DEFAULT_PERMISSION: Lazy = Lazy::new(|| {
+ ChannelPermission::SendEmbeds
+ ChannelPermission::UploadFiles
+ ChannelPermission::Connect
- + ChannelPermission::Speak,
+ + ChannelPermission::Speak
+ + ChannelPermission::Listen
+ + ChannelPermission::Video
)
});
pub static DEFAULT_PERMISSION_SAVED_MESSAGES: u64 = ChannelPermission::GrantAllSafe as u64;
pub static DEFAULT_PERMISSION_DIRECT_MESSAGE: Lazy = Lazy::new(|| {
- DEFAULT_PERMISSION.add(ChannelPermission::ManageChannel + ChannelPermission::React)
+ DEFAULT_PERMISSION.add(ChannelPermission::ManageChannel + ChannelPermission::React + ChannelPermission::Masquerade)
});
pub static DEFAULT_PERMISSION_SERVER: Lazy = Lazy::new(|| {
diff --git a/crates/core/permissions/src/models/server.rs b/crates/core/permissions/src/models/server.rs
index e6355eedc..8391029a0 100644
--- a/crates/core/permissions/src/models/server.rs
+++ b/crates/core/permissions/src/models/server.rs
@@ -39,7 +39,7 @@ pub enum DataPermissionPoly {
permissions: u64,
},
Field {
- /// Allow / deny values to set for members in this `TextChannel` or `VoiceChannel`
+ /// Allow / deny values to set for members in this server channel
permissions: Override,
},
}
diff --git a/crates/core/permissions/src/test.rs b/crates/core/permissions/src/test.rs
index e4dadd485..93e1dea2c 100644
--- a/crates/core/permissions/src/test.rs
+++ b/crates/core/permissions/src/test.rs
@@ -64,6 +64,14 @@ async fn validate_user_permissions() {
unreachable!()
}
+ async fn do_we_have_publish_overwrites(&mut self) -> bool {
+ true
+ }
+
+ async fn do_we_have_receive_overwrites(&mut self) -> bool {
+ true
+ }
+
async fn get_channel_type(&mut self) -> ChannelType {
ChannelType::DirectMessage
}
@@ -153,6 +161,14 @@ async fn validate_group_permissions() {
unreachable!()
}
+ async fn do_we_have_publish_overwrites(&mut self) -> bool {
+ true
+ }
+
+ async fn do_we_have_receive_overwrites(&mut self) -> bool {
+ true
+ }
+
async fn get_channel_type(&mut self) -> ChannelType {
ChannelType::Group
}
@@ -254,6 +270,14 @@ async fn validate_server_permissions() {
false
}
+ async fn do_we_have_publish_overwrites(&mut self) -> bool {
+ true
+ }
+
+ async fn do_we_have_receive_overwrites(&mut self) -> bool {
+ true
+ }
+
async fn get_channel_type(&mut self) -> ChannelType {
ChannelType::ServerChannel
}
@@ -346,6 +370,14 @@ async fn validate_timed_out_member() {
true
}
+ async fn do_we_have_publish_overwrites(&mut self) -> bool {
+ true
+ }
+
+ async fn do_we_have_receive_overwrites(&mut self) -> bool {
+ true
+ }
+
async fn get_channel_type(&mut self) -> ChannelType {
ChannelType::ServerChannel
}
diff --git a/crates/core/permissions/src/trait.rs b/crates/core/permissions/src/trait.rs
index d4a7c3d36..96b42caf8 100644
--- a/crates/core/permissions/src/trait.rs
+++ b/crates/core/permissions/src/trait.rs
@@ -39,6 +39,12 @@ pub trait PermissionQuery {
/// Is our perspective user timed out on this server?
async fn are_we_timed_out(&mut self) -> bool;
+ /// Is the member muted?
+ async fn do_we_have_publish_overwrites(&mut self) -> bool;
+
+ /// Is the member deafend?
+ async fn do_we_have_receive_overwrites(&mut self) -> bool;
+
// * For calculating channel permission
/// Get the type of the channel
diff --git a/crates/core/presence/Cargo.toml b/crates/core/presence/Cargo.toml
index 2e53c3e70..7868342d3 100644
--- a/crates/core/presence/Cargo.toml
+++ b/crates/core/presence/Cargo.toml
@@ -1,10 +1,11 @@
[package]
name = "revolt-presence"
-version = "0.8.8"
+version = "0.13.7"
edition = "2021"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles "]
description = "Revolt Backend: User Presence"
+repository = "https://github.com/stoatchat/stoatchat"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -13,16 +14,16 @@ redis-is-patched = []
[dev-dependencies]
# Async
-async-std = { version = "1.8.0", features = ["attributes"] }
+async-std = { workspace = true, features = ["attributes"] }
# Config for loading Redis URI
-revolt-config = { version = "0.8.8", path = "../config" }
+revolt-config = { workspace = true }
[dependencies]
# Utility
-log = "0.4.17"
-rand = "0.8.5"
-once_cell = "1.17.1"
+log = { workspace = true }
+rand = { workspace = true }
+once_cell = { workspace = true }
# Redis
-redis-kiss = "0.1.4"
+redis-kiss = { workspace = true }
diff --git a/crates/core/ratelimits/Cargo.toml b/crates/core/ratelimits/Cargo.toml
new file mode 100644
index 000000000..e606d151f
--- /dev/null
+++ b/crates/core/ratelimits/Cargo.toml
@@ -0,0 +1,34 @@
+[package]
+name = "revolt-ratelimits"
+version = "0.13.7"
+edition = "2024"
+license = "MIT"
+authors = ["Zomatree ", "Paul Makles "]
+description = "Revolt Backend: Ratelimit Handler"
+repository = "https://github.com/stoatchat/stoatchat"
+
+[features]
+rocket = [
+ "dep:rocket",
+ "dep:revolt_rocket_okapi",
+ "revolt-database/rocket-impl",
+]
+axum = ["dep:axum", "revolt-database/axum-impl"]
+
+default = ["rocket", "axum"]
+
+[dependencies]
+revolt-database = { workspace = true }
+revolt-result = { workspace = true }
+revolt-config = { workspace = true }
+
+rocket = { workspace = true, optional = true }
+revolt_rocket_okapi = { workspace = true, optional = true }
+
+axum = { workspace = true, optional = true, features = ["macros"] }
+
+serde = { workspace = true }
+authifier = { workspace = true }
+dashmap = { workspace = true }
+async-trait = { workspace = true }
+log = { workspace = true }
diff --git a/crates/core/ratelimits/src/axum.rs b/crates/core/ratelimits/src/axum.rs
new file mode 100644
index 000000000..f50cf8671
--- /dev/null
+++ b/crates/core/ratelimits/src/axum.rs
@@ -0,0 +1,194 @@
+use std::net::SocketAddr;
+
+use async_trait::async_trait;
+use axum::{
+ Json, RequestPartsExt, Router,
+ body::Body,
+ extract::{ConnectInfo, FromRef, FromRequestParts, State},
+ http::{HeaderValue, Request, StatusCode, request::Parts},
+ middleware::Next,
+ response::{IntoResponse, Response},
+ routing::get,
+};
+use revolt_database::{Database, User};
+use revolt_config::config;
+
+use crate::ratelimiter::{RatelimitInformation, Ratelimiter, RequestKind};
+
+#[derive(Clone, Copy)]
+pub struct AxumRequestKind;
+
+impl RequestKind for AxumRequestKind {
+ type R<'a> = Parts;
+}
+
+pub type RatelimitStorage = crate::ratelimiter::RatelimitStorage;
+
+fn to_ip(parts: &Parts) -> String {
+ parts
+ .extensions
+ .get::>()
+ .map(|info| info.ip().to_string())
+ .unwrap_or_default()
+}
+
+async fn to_real_ip(parts: &Parts) -> String {
+ if config().await.api.security.trust_cloudflare {
+ parts
+ .headers
+ .get("CF-Connecting-IP")
+ .map(|x| x.to_str().unwrap().to_string())
+ .unwrap_or_else(|| to_ip(parts))
+ } else {
+ to_ip(parts)
+ }
+}
+
+#[async_trait]
+impl FromRequestParts
for Ratelimiter
+where
+ Database: FromRef,
+ RatelimitStorage: FromRef,
+{
+ type Rejection = Json;
+
+ async fn from_request_parts(parts: &mut Parts, state: &S) -> Result {
+ if parts
+ .extensions
+ .get::>>()
+ .is_none()
+ {
+ let storage = RatelimitStorage::from_ref(state);
+
+ let identifier = if let Ok(user) = parts.extract_with_state::(state).await {
+ user.id
+ } else {
+ to_real_ip(parts).await
+ };
+
+ let (bucket, resource) = storage.resolver.resolve_bucket(parts);
+ let limit = storage.resolver.resolve_bucket_limit(bucket);
+
+ let ratelimiter =
+ Ratelimiter::from(&storage.map, &identifier, limit, (bucket, resource));
+
+ parts.extensions.insert(ratelimiter.map_err(Json));
+ };
+
+ *parts
+ .extensions
+ .get::>>()
+ .unwrap()
+ }
+}
+
+#[async_trait]
+impl FromRequestParts for RatelimitInformation
+where
+ Database: FromRef,
+ RatelimitStorage: FromRef,
+{
+ type Rejection = Json;
+
+ async fn from_request_parts(parts: &mut Parts, state: &S) -> Result {
+ if parts
+ .extensions
+ .get::>>()
+ .is_none()
+ {
+ let ratelimiter = parts.extract_with_state::(state).await;
+
+ parts.extensions.insert(ratelimiter);
+ };
+
+ let ratelimiter = *parts
+ .extensions
+ .get::>>()
+ .unwrap();
+
+ match ratelimiter {
+ Ok(ratelimter) => Ok(RatelimitInformation::Success(ratelimter)),
+ Err(ratelimiter) => Err(Json(RatelimitInformation::Failure {
+ retry_after: ratelimiter.reset,
+ })),
+ }
+ }
+}
+
+pub async fn ratelimit_middleware(
+ State(database): State,
+ State(ratelimit_storage): State,
+ request: Request,
+ next: Next,
+) -> Response {
+ #[derive(axum::extract::FromRef)]
+ struct TempState {
+ database: Database,
+ ratelimit_storage: RatelimitStorage,
+ }
+
+ let state = TempState {
+ database,
+ ratelimit_storage,
+ };
+
+ let (mut parts, body) = request.into_parts();
+
+ let res = Ratelimiter::from_request_parts(&mut parts, &state).await;
+
+ let (Ok(ratelimiter) | Err(Json(ratelimiter))) = &res;
+
+ let mut response = if res.is_ok() {
+ let request = Request::from_parts(parts, body);
+
+ next.run(request).await
+ } else {
+ let ratelimit_info = RatelimitInformation::from_request_parts(&mut parts, &state).await;
+
+ ratelimit_info.map(Json).into_response()
+ };
+
+ let Ratelimiter {
+ key,
+ limit,
+ remaining,
+ reset,
+ } = ratelimiter;
+
+ let headers = response.headers_mut();
+
+ headers.insert(
+ "X-RateLimit-Limit",
+ HeaderValue::from_str(&limit.to_string()).unwrap(),
+ );
+ headers.insert(
+ "X-RateLimit-Bucket",
+ HeaderValue::from_str(&key.to_string()).unwrap(),
+ );
+ headers.insert(
+ "X-RateLimit-Remaining",
+ HeaderValue::from_str(&remaining.to_string()).unwrap(),
+ );
+ headers.insert(
+ "X-RateLimit-Reset-After",
+ HeaderValue::from_str(&reset.to_string()).unwrap(),
+ );
+
+ if res.is_err() {
+ *response.status_mut() = StatusCode::TOO_MANY_REQUESTS;
+ };
+
+ response
+}
+
+async fn ratelimit_info(info: RatelimitInformation) -> Json {
+ Json(info)
+}
+
+pub fn routes() -> Router
+where
+ Database: FromRef,
+ RatelimitStorage: FromRef,
+{
+ Router::new().route("/ratelimit", get(ratelimit_info))
+}
diff --git a/crates/core/ratelimits/src/lib.rs b/crates/core/ratelimits/src/lib.rs
new file mode 100644
index 000000000..5eaa81080
--- /dev/null
+++ b/crates/core/ratelimits/src/lib.rs
@@ -0,0 +1,7 @@
+pub mod ratelimiter;
+
+#[cfg(feature = "rocket")]
+pub mod rocket;
+
+#[cfg(feature = "axum")]
+pub mod axum;
diff --git a/crates/core/ratelimits/src/ratelimiter.rs b/crates/core/ratelimits/src/ratelimiter.rs
new file mode 100644
index 000000000..a232feb32
--- /dev/null
+++ b/crates/core/ratelimits/src/ratelimiter.rs
@@ -0,0 +1,145 @@
+use std::collections::hash_map::DefaultHasher;
+use std::hash::Hasher;
+use std::ops::Add;
+use std::sync::Arc;
+use std::time::{Duration, SystemTime, UNIX_EPOCH};
+
+use serde::Serialize;
+
+use dashmap::DashMap;
+
+pub trait RequestKind {
+ type R<'a>;
+}
+
+pub trait RatelimitResolver: Send + Sync {
+ fn resolve_bucket<'a>(&self, request: &'a R) -> (&'a str, Option<&'a str>);
+ fn resolve_bucket_limit(&self, bucket: &str) -> u32;
+}
+
+#[derive(Clone)]
+pub struct RatelimitStorage {
+ pub resolver: Arc RatelimitResolver>>,
+ pub map: Arc>,
+}
+
+impl RatelimitStorage {
+ pub fn new RatelimitResolver> + 'static>(resolver: R) -> Self {
+ Self {
+ resolver: Arc::new(resolver),
+ map: Arc::new(DashMap::new()),
+ }
+ }
+}
+
+/// Ratelimit Bucket
+#[derive(Clone, Copy, Debug)]
+pub struct Entry {
+ used: u32,
+ reset: u128,
+}
+
+/// Get the current time from Unix Epoch as a Duration
+fn now() -> Duration {
+ SystemTime::now()
+ .duration_since(UNIX_EPOCH)
+ .expect("Time went backwards...")
+}
+
+impl Entry {
+ /// Find bucket by its key
+ pub fn from(map: &DashMap, key: u64) -> Entry {
+ map.get(&key).map(|x| *x).unwrap_or_else(|| Entry {
+ used: 0,
+ reset: now().add(Duration::from_secs(10)).as_millis(),
+ })
+ }
+
+ /// Deduct one unit from the bucket and save
+ pub fn deduct(&mut self) {
+ let current_time = now().as_millis();
+ if current_time > self.reset {
+ self.used = 1;
+ self.reset = now().add(Duration::from_secs(10)).as_millis();
+ } else {
+ self.used += 1;
+ }
+ }
+
+ /// Save information
+ pub fn save(self, map: &DashMap, key: u64) {
+ map.insert(key, self);
+ }
+
+ /// Get remaining units in the bucket
+ pub fn get_remaining(&self, limit: u32) -> u32 {
+ if now().as_millis() > self.reset {
+ limit
+ } else {
+ limit - self.used
+ }
+ }
+
+ /// Get how long bucket has until reset
+ pub fn left_until_reset(&self) -> u128 {
+ let current_time = now().as_millis();
+ self.reset.saturating_sub(current_time)
+ }
+}
+
+/// Ratelimit Guard
+#[derive(Serialize, Clone, Copy, Debug)]
+#[allow(dead_code)]
+pub struct Ratelimiter {
+ pub key: u64,
+ pub limit: u32,
+ pub remaining: u32,
+ pub reset: u128,
+}
+
+impl Ratelimiter {
+ /// Generate guard from identifier and target bucket
+ pub fn from(
+ map: &DashMap,
+ identifier: &str,
+ limit: u32,
+ (bucket, resource): (&str, Option<&str>),
+ ) -> Result {
+ let mut key = DefaultHasher::new();
+ key.write(identifier.as_bytes());
+ key.write(bucket.as_bytes());
+
+ if let Some(id) = resource {
+ key.write(id.as_bytes());
+ }
+
+ let key = key.finish();
+ let mut entry = Entry::from(map, key);
+
+ let remaining = entry.get_remaining(limit);
+ let reset = entry.left_until_reset();
+ let mut ratelimiter = Ratelimiter {
+ key,
+ limit,
+ remaining,
+ reset,
+ };
+ if remaining == 0 {
+ return Err(ratelimiter);
+ }
+
+ entry.deduct();
+ entry.save(map, key);
+ ratelimiter.remaining -= 1;
+ ratelimiter.reset = entry.left_until_reset();
+
+ Ok(ratelimiter)
+ }
+}
+
+#[derive(Serialize)]
+#[serde(untagged)]
+pub enum RatelimitInformation {
+ Success(Ratelimiter),
+ Failure { retry_after: u128 },
+}
diff --git a/crates/core/ratelimits/src/rocket.rs b/crates/core/ratelimits/src/rocket.rs
new file mode 100644
index 000000000..f790abb9f
--- /dev/null
+++ b/crates/core/ratelimits/src/rocket.rs
@@ -0,0 +1,163 @@
+use async_trait::async_trait;
+use log::info;
+use revolt_config::config;
+use rocket::fairing::{Fairing, Info, Kind};
+use rocket::http::uri::Origin;
+use rocket::http::{Method, Status};
+use rocket::request::{FromRequest, Outcome};
+use rocket::serde::json::Json;
+use rocket::{Data, Request, Response, State};
+
+use revolt_rocket_okapi::r#gen::OpenApiGenerator;
+use revolt_rocket_okapi::request::{OpenApiFromRequest, RequestHeaderInput};
+
+use authifier::models::Session;
+
+use crate::ratelimiter::RequestKind;
+use crate::ratelimiter::{RatelimitInformation, Ratelimiter};
+
+#[derive(Clone, Copy)]
+pub struct RocketRequestKind;
+
+impl RequestKind for RocketRequestKind {
+ type R<'a> = Request<'a>;
+}
+
+pub type RatelimitStorage = crate::ratelimiter::RatelimitStorage;
+
+/// Find the remote IP of the client
+fn to_ip(request: &'_ rocket::Request<'_>) -> String {
+ request
+ .client_ip()
+ .map(|r| r.to_string())
+ .unwrap_or_default()
+}
+
+/// Find the actual IP of the client
+async fn to_real_ip(request: &'_ rocket::Request<'_>) -> String {
+ if config().await.api.security.trust_cloudflare {
+ request
+ .headers()
+ .get_one("CF-Connecting-IP")
+ .map(|x| x.to_string())
+ .unwrap_or_else(|| to_ip(request))
+ } else {
+ to_ip(request)
+ }
+}
+
+#[async_trait]
+impl<'r> FromRequest<'r> for Ratelimiter {
+ type Error = Ratelimiter;
+
+ async fn from_request<'a>(request: &'r rocket::Request<'a>) -> Outcome {
+ let ratelimiter = request
+ .local_cache_async(async {
+ use rocket::outcome::Outcome;
+
+ let storage = request.guard::<&State>().await.unwrap();
+
+ let identifier = if let Outcome::Success(session) = request.guard::().await
+ {
+ session.id
+ } else {
+ to_real_ip(request).await
+ };
+
+ let (bucket, resource) = storage.resolver.resolve_bucket(request);
+ let limit = storage.resolver.resolve_bucket_limit(bucket);
+
+ Ratelimiter::from(&storage.map, &identifier, limit, (bucket, resource))
+ })
+ .await;
+
+ match ratelimiter {
+ Ok(ratelimiter) => Outcome::Success(*ratelimiter),
+ Err(ratelimiter) => Outcome::Error((Status::TooManyRequests, *ratelimiter)),
+ }
+ }
+}
+
+impl OpenApiFromRequest<'_> for Ratelimiter {
+ fn from_request_input(
+ _gen: &mut OpenApiGenerator,
+ _name: String,
+ _required: bool,
+ ) -> revolt_rocket_okapi::Result {
+ Ok(RequestHeaderInput::None)
+ }
+}
+
+/// Attach ratelimiter to the Rocket application
+pub struct RatelimitFairing;
+
+#[async_trait]
+impl Fairing for RatelimitFairing {
+ fn info(&self) -> Info {
+ Info {
+ name: "Ratelimiter",
+ kind: Kind::Request | Kind::Response,
+ }
+ }
+
+ async fn on_request(&self, request: &mut Request<'_>, _: &mut Data<'_>) {
+ use rocket::outcome::Outcome;
+ if let Outcome::Error(_) = request.guard::().await {
+ info!(
+ "User rate-limited on route {}! (IP = {:?})",
+ request.uri(),
+ to_real_ip(request).await
+ );
+
+ request.set_method(Method::Get);
+ request.set_uri(Origin::parse("/ratelimit").unwrap())
+ }
+ }
+
+ async fn on_response<'r>(&self, request: &'r Request<'_>, response: &mut Response<'r>) {
+ let guard = request.guard::().await;
+ let (Outcome::Success(ratelimiter) | Outcome::Error((_, ratelimiter))) = guard else {
+ unreachable!()
+ };
+ let Ratelimiter {
+ key,
+ limit,
+ remaining,
+ reset,
+ } = ratelimiter;
+
+ response.set_raw_header("X-RateLimit-Limit", limit.to_string());
+ response.set_raw_header("X-RateLimit-Bucket", key.to_string());
+ response.set_raw_header("X-RateLimit-Remaining", remaining.to_string());
+ response.set_raw_header("X-RateLimit-Reset-After", reset.to_string());
+
+ if guard.is_error() {
+ response.set_status(Status::TooManyRequests);
+ }
+ }
+}
+
+#[async_trait]
+impl<'r> FromRequest<'r> for RatelimitInformation {
+ type Error = u128;
+
+ async fn from_request(request: &'r rocket::Request<'_>) -> Outcome {
+ let info = match request.guard::().await {
+ Outcome::Success(ratelimiter) => RatelimitInformation::Success(ratelimiter),
+ Outcome::Error((_, ratelimiter)) => RatelimitInformation::Failure {
+ retry_after: ratelimiter.reset,
+ },
+ _ => unreachable!(),
+ };
+ Outcome::Success(info)
+ }
+}
+
+#[rocket::get("/ratelimit")]
+fn ratelimit_info(info: RatelimitInformation) -> Json {
+ Json(info)
+}
+
+pub fn routes() -> Vec {
+ rocket::routes![ratelimit_info]
+}
diff --git a/crates/core/result/Cargo.toml b/crates/core/result/Cargo.toml
index 3a25d206f..9cc5db3f7 100644
--- a/crates/core/result/Cargo.toml
+++ b/crates/core/result/Cargo.toml
@@ -1,10 +1,11 @@
[package]
name = "revolt-result"
-version = "0.8.8"
+version = "0.13.7"
edition = "2021"
license = "MIT"
authors = ["Paul Makles "]
description = "Revolt Backend: Result and Error types"
+repository = "https://github.com/stoatchat/stoatchat"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -15,22 +16,27 @@ utoipa = ["dep:utoipa"]
rocket = ["dep:rocket", "dep:serde_json"]
axum = ["dep:axum", "dep:serde_json"]
okapi = ["dep:revolt_rocket_okapi", "dep:revolt_okapi", "schemas"]
+sentry = ["dep:sentry"]
-default = ["serde"]
+default = ["serde", "sentry"]
[dependencies]
# Serialisation
-serde_json = { version = "1", optional = true }
-serde = { version = "1", features = ["derive"], optional = true }
+serde_json = { workspace = true, optional = true }
+serde = { workspace = true, optional = true }
# Spec Generation
-schemars = { version = "0.8.8", optional = true }
-utoipa = { version = "4.2.3", optional = true }
+schemars = { workspace = true, optional = true }
+utoipa = { workspace = true, optional = true }
# Rocket
-rocket = { optional = true, version = "0.5.0-rc.2", default-features = false }
-revolt_rocket_okapi = { version = "0.10.0", optional = true }
-revolt_okapi = { version = "0.9.1", optional = true }
+rocket = { workspace = true, optional = true }
+revolt_rocket_okapi = { workspace = true, optional = true }
+revolt_okapi = { workspace = true, optional = true }
+# utilities
+log = { workspace = true }
# Axum
-axum = { version = "0.7.5", optional = true }
+axum = { workspace = true, optional = true }
+
+sentry = { workspace = true, optional = true }
diff --git a/crates/core/result/src/axum.rs b/crates/core/result/src/axum.rs
index e4caf8183..440c4fc4b 100644
--- a/crates/core/result/src/axum.rs
+++ b/crates/core/result/src/axum.rs
@@ -24,6 +24,7 @@ impl IntoResponse for Error {
ErrorType::UnknownChannel => StatusCode::NOT_FOUND,
ErrorType::UnknownMessage => StatusCode::NOT_FOUND,
ErrorType::UnknownAttachment => StatusCode::BAD_REQUEST,
+ ErrorType::CannotDeleteMessage => StatusCode::FORBIDDEN,
ErrorType::CannotEditMessage => StatusCode::FORBIDDEN,
ErrorType::CannotJoinCall => StatusCode::BAD_REQUEST,
ErrorType::TooManyAttachments { .. } => StatusCode::BAD_REQUEST,
@@ -36,7 +37,9 @@ impl IntoResponse for Error {
ErrorType::NotInGroup => StatusCode::NOT_FOUND,
ErrorType::AlreadyPinned => StatusCode::BAD_REQUEST,
ErrorType::NotPinned => StatusCode::BAD_REQUEST,
+ ErrorType::InSlowmode { retry_after: _ } => StatusCode::TOO_MANY_REQUESTS,
+ ErrorType::CantCreateServers => StatusCode::FORBIDDEN,
ErrorType::UnknownServer => StatusCode::NOT_FOUND,
ErrorType::InvalidRole => StatusCode::NOT_FOUND,
ErrorType::Banned => StatusCode::FORBIDDEN,
@@ -62,6 +65,7 @@ impl IntoResponse for Error {
ErrorType::NotPrivileged => StatusCode::FORBIDDEN,
ErrorType::CannotGiveMissingPermissions => StatusCode::FORBIDDEN,
ErrorType::NotOwner => StatusCode::FORBIDDEN,
+ ErrorType::IsElevated => StatusCode::FORBIDDEN,
ErrorType::DatabaseError { .. } => StatusCode::INTERNAL_SERVER_ERROR,
ErrorType::InternalError => StatusCode::INTERNAL_SERVER_ERROR,
@@ -73,8 +77,13 @@ impl IntoResponse for Error {
ErrorType::DuplicateNonce => StatusCode::CONFLICT,
ErrorType::VosoUnavailable => StatusCode::BAD_REQUEST,
ErrorType::NotFound => StatusCode::NOT_FOUND,
- ErrorType::NoEffect => StatusCode::OK,
+ ErrorType::NoEffect => StatusCode::BAD_REQUEST,
ErrorType::FailedValidation { .. } => StatusCode::BAD_REQUEST,
+ ErrorType::LiveKitUnavailable => StatusCode::BAD_REQUEST,
+ ErrorType::NotConnected => StatusCode::BAD_REQUEST,
+ ErrorType::NotAVoiceChannel => StatusCode::BAD_REQUEST,
+ ErrorType::AlreadyConnected => StatusCode::BAD_REQUEST,
+ ErrorType::UnknownNode => StatusCode::BAD_REQUEST,
ErrorType::InvalidFlagValue => StatusCode::BAD_REQUEST,
ErrorType::FeatureDisabled { .. } => StatusCode::BAD_REQUEST,
diff --git a/crates/core/result/src/lib.rs b/crates/core/result/src/lib.rs
index 10b56d5d4..4124a0476 100644
--- a/crates/core/result/src/lib.rs
+++ b/crates/core/result/src/lib.rs
@@ -1,4 +1,5 @@
use std::fmt::Display;
+use std::panic::Location;
#[cfg(feature = "serde")]
#[macro_use]
@@ -77,6 +78,7 @@ pub enum ErrorType {
UnknownChannel,
UnknownAttachment,
UnknownMessage,
+ CannotDeleteMessage,
CannotEditMessage,
CannotJoinCall,
TooManyAttachments {
@@ -101,8 +103,12 @@ pub enum ErrorType {
NotInGroup,
AlreadyPinned,
NotPinned,
+ InSlowmode {
+ retry_after: u64,
+ },
// ? Server related errors
+ CantCreateServers,
UnknownServer,
InvalidRole,
Banned,
@@ -138,6 +144,7 @@ pub enum ErrorType {
NotPrivileged,
CannotGiveMissingPermissions,
NotOwner,
+ IsElevated,
// ? General errors
DatabaseError {
@@ -158,6 +165,12 @@ pub enum ErrorType {
error: String,
},
+ // ? Voice errors
+ LiveKitUnavailable,
+ NotAVoiceChannel,
+ AlreadyConnected,
+ NotConnected,
+ UnknownNode,
// ? Micro-service errors
ProxyError,
FileTooSmall,
@@ -197,6 +210,58 @@ macro_rules! create_database_error {
};
}
+#[macro_export]
+#[cfg(debug_assertions)]
+macro_rules! query {
+ ( $self: ident, $type: ident, $collection: expr, $($rest:expr),+ ) => {
+ Ok($self.$type($collection, $($rest),+).await.unwrap())
+ };
+}
+
+#[macro_export]
+#[cfg(not(debug_assertions))]
+macro_rules! query {
+ ( $self: ident, $type: ident, $collection: expr, $($rest:expr),+ ) => {
+ $self.$type($collection, $($rest),+).await
+ .map_err(|_| create_database_error!(stringify!($type), $collection))
+ };
+}
+
+pub trait ToRevoltError {
+ #[track_caller]
+ fn to_internal_error(self) -> Result;
+}
+
+impl ToRevoltError for Result {
+ #[track_caller]
+ fn to_internal_error(self) -> Result {
+ let loc = Location::caller();
+
+ self.map_err(|e| {
+ log::error!("{e:?}");
+ #[cfg(feature = "sentry")]
+ sentry::capture_error(&e);
+
+ Error {
+ error_type: ErrorType::InternalError,
+ location: format!("{}:{}:{}", loc.file(), loc.line(), loc.column()),
+ }
+ })
+ }
+}
+
+impl ToRevoltError for Option {
+ #[track_caller]
+ fn to_internal_error(self) -> Result {
+ let loc = Location::caller();
+
+ self.ok_or_else(|| Error {
+ error_type: ErrorType::InternalError,
+ location: format!("{}:{}:{}", loc.file(), loc.line(), loc.column()),
+ })
+ }
+}
+
#[cfg(test)]
mod tests {
use crate::ErrorType;
diff --git a/crates/core/result/src/rocket.rs b/crates/core/result/src/rocket.rs
index 1d996a173..76719149a 100644
--- a/crates/core/result/src/rocket.rs
+++ b/crates/core/result/src/rocket.rs
@@ -30,6 +30,7 @@ impl<'r> Responder<'r, 'static> for Error {
ErrorType::UnknownChannel => Status::NotFound,
ErrorType::UnknownMessage => Status::NotFound,
ErrorType::UnknownAttachment => Status::BadRequest,
+ ErrorType::CannotDeleteMessage => Status::Forbidden,
ErrorType::CannotEditMessage => Status::Forbidden,
ErrorType::CannotJoinCall => Status::BadRequest,
ErrorType::TooManyAttachments { .. } => Status::BadRequest,
@@ -42,8 +43,10 @@ impl<'r> Responder<'r, 'static> for Error {
ErrorType::NotInGroup => Status::NotFound,
ErrorType::AlreadyPinned => Status::BadRequest,
ErrorType::NotPinned => Status::BadRequest,
+ ErrorType::InSlowmode { retry_after: _ } => Status::TooManyRequests,
ErrorType::InvalidFlagValue => Status::BadRequest,
+ ErrorType::CantCreateServers => Status::Forbidden,
ErrorType::UnknownServer => Status::NotFound,
ErrorType::InvalidRole => Status::NotFound,
ErrorType::Banned => Status::Forbidden,
@@ -69,6 +72,7 @@ impl<'r> Responder<'r, 'static> for Error {
ErrorType::NotPrivileged => Status::Forbidden,
ErrorType::CannotGiveMissingPermissions => Status::Forbidden,
ErrorType::NotOwner => Status::Forbidden,
+ ErrorType::IsElevated => Status::Forbidden,
ErrorType::DatabaseError { .. } => Status::InternalServerError,
ErrorType::InternalError => Status::InternalServerError,
@@ -78,10 +82,14 @@ impl<'r> Responder<'r, 'static> for Error {
ErrorType::InvalidSession => Status::Unauthorized,
ErrorType::NotAuthenticated => Status::Unauthorized,
ErrorType::DuplicateNonce => Status::Conflict,
- ErrorType::VosoUnavailable => Status::BadRequest,
ErrorType::NotFound => Status::NotFound,
- ErrorType::NoEffect => Status::Ok,
+ ErrorType::NoEffect => Status::BadRequest,
ErrorType::FailedValidation { .. } => Status::BadRequest,
+ ErrorType::LiveKitUnavailable => Status::BadRequest,
+ ErrorType::NotAVoiceChannel => Status::BadRequest,
+ ErrorType::AlreadyConnected => Status::BadRequest,
+ ErrorType::NotConnected => Status::BadRequest,
+ ErrorType::UnknownNode => Status::BadRequest,
ErrorType::FeatureDisabled { .. } => Status::BadRequest,
ErrorType::ProxyError => Status::BadRequest,
@@ -90,6 +98,7 @@ impl<'r> Responder<'r, 'static> for Error {
ErrorType::FileTypeNotAllowed => Status::BadRequest,
ErrorType::ImageProcessingFailed => Status::InternalServerError,
ErrorType::NoEmbedData => Status::BadRequest,
+ ErrorType::VosoUnavailable => Status::BadRequest,
};
// Serialize the error data structure into JSON.
diff --git a/crates/daemons/crond/Cargo.toml b/crates/daemons/crond/Cargo.toml
index 2c55f2f76..bb2f6ced7 100644
--- a/crates/daemons/crond/Cargo.toml
+++ b/crates/daemons/crond/Cargo.toml
@@ -1,22 +1,37 @@
[package]
name = "revolt-crond"
-version = "0.8.8"
+version = "0.13.7"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles "]
edition = "2021"
description = "Revolt Daemon Service: Timed data clean up tasks"
+publish = false
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
# Utility
-log = "0.4"
+log = { workspace = true }
# Async
-tokio = { version = "1" }
+tokio = { workspace = true }
+
+# Redis
+redis-kiss = { workspace = true }
+
+# RabbitMQ
+lapin = { workspace = true }
+futures-lite = { workspace = true }
+
+# Processing
+serde_json = { workspace = true }
+revolt_optional_struct = { workspace = true }
+serde = { workspace = true }
+iso8601-timestamp = { workspace = true, features = ["serde", "bson"] }
# Core
-revolt-database = { version = "0.8.8", path = "../../core/database" }
-revolt-result = { version = "0.8.8", path = "../../core/result" }
-revolt-config = { version = "0.8.8", path = "../../core/config" }
-revolt-files = { version = "0.8.8", path = "../../core/files" }
+revolt-database = { workspace = true }
+revolt-result = { workspace = true }
+revolt-config = { workspace = true }
+revolt-files = { workspace = true }
+revolt-permissions = { workspace = true }
diff --git a/crates/daemons/crond/Dockerfile b/crates/daemons/crond/Dockerfile
index 8a4c915a9..f18524573 100644
--- a/crates/daemons/crond/Dockerfile
+++ b/crates/daemons/crond/Dockerfile
@@ -1,10 +1,11 @@
# Build Stage
-FROM ghcr.io/revoltchat/base:latest AS builder
+FROM ghcr.io/stoatchat/base:latest AS builder
FROM debian:12 AS debian
# Bundle Stage
FROM gcr.io/distroless/cc-debian12:nonroot
COPY --from=builder /home/rust/src/target/release/revolt-crond ./
+COPY --from=debian /usr/bin/uname /usr/bin/uname
USER nonroot
CMD ["./revolt-crond"]
\ No newline at end of file
diff --git a/crates/daemons/crond/src/main.rs b/crates/daemons/crond/src/main.rs
index b232a10a6..2b9a82f65 100644
--- a/crates/daemons/crond/src/main.rs
+++ b/crates/daemons/crond/src/main.rs
@@ -1,7 +1,7 @@
use revolt_config::configure;
-use revolt_database::DatabaseInfo;
+use revolt_database::{DatabaseInfo, AMQP};
use revolt_result::Result;
-use tasks::{file_deletion, prune_dangling_files, prune_members};
+use tasks::{acks, file_deletion, prune_dangling_files, prune_members};
use tokio::try_join;
pub mod tasks;
@@ -11,10 +11,13 @@ async fn main() -> Result<()> {
configure!(crond);
let db = DatabaseInfo::Auto.connect().await.expect("database");
+ let amqp = AMQP::new_auto().await;
+
try_join!(
file_deletion::task(db.clone()),
prune_dangling_files::task(db.clone()),
- prune_members::task(db.clone())
+ prune_members::task(db.clone()),
+ acks::task(db.clone(), amqp.clone()),
)
.map(|_| ())
}
diff --git a/crates/daemons/crond/src/tasks/acks.rs b/crates/daemons/crond/src/tasks/acks.rs
new file mode 100644
index 000000000..7576bac9b
--- /dev/null
+++ b/crates/daemons/crond/src/tasks/acks.rs
@@ -0,0 +1,164 @@
+use futures_lite::stream::StreamExt;
+use lapin::{
+ options::*,
+ types::FieldTable,
+ uri::{AMQPAuthority, AMQPQueryString, AMQPUri, AMQPUserInfo},
+ ConnectionBuilder, ConnectionProperties, ExchangeKind,
+};
+use log::{debug, info};
+use redis_kiss::{get_connection, AsyncCommands, Conn as RedisConnection};
+use revolt_config::config;
+use revolt_database::{events::rabbit::AckEventPayload, Database, AMQP};
+use revolt_result::{Result, ToRevoltError};
+use serde_json;
+
+pub async fn task(db: Database, amqp: AMQP) -> Result<()> {
+ let config = config().await;
+
+ let mut redis = get_connection()
+ .await
+ .expect("Failed to get redis connection");
+
+ let uri = AMQPUri {
+ scheme: lapin::uri::AMQPScheme::AMQP,
+ authority: AMQPAuthority {
+ userinfo: AMQPUserInfo {
+ username: config.rabbit.username,
+ password: config.rabbit.password,
+ },
+ host: config.rabbit.host,
+ port: config.rabbit.port,
+ },
+ vhost: "/".to_string(),
+ query: AMQPQueryString::default(),
+ };
+
+ let connection = ConnectionBuilder::new()
+ .expect("Builder")
+ .with_uri(uri)
+ .with_properties(ConnectionProperties::default())
+ .connect()
+ .await
+ .expect("Failed to connect to rabbitmq");
+
+ let reader_channel = connection
+ .create_channel()
+ .await
+ .expect("Failed to create channel");
+
+ reader_channel
+ .exchange_declare(
+ config.rabbit.default_exchange.clone().into(),
+ ExchangeKind::Topic,
+ ExchangeDeclareOptions {
+ durable: true,
+ ..Default::default()
+ },
+ FieldTable::default(),
+ )
+ .await
+ .expect("Failed to declare exchange");
+
+ reader_channel
+ .queue_declare(
+ config.rabbit.queues.acks.clone().into(),
+ QueueDeclareOptions {
+ durable: true,
+ ..Default::default()
+ },
+ FieldTable::default(),
+ )
+ .await
+ .expect("Failed to bind queue");
+
+ reader_channel
+ .queue_bind(
+ config.rabbit.queues.acks.clone().into(),
+ config.rabbit.default_exchange.into(),
+ config.rabbit.queues.acks.clone().into(),
+ QueueBindOptions::default(),
+ FieldTable::default(),
+ )
+ .await
+ .expect("Failed to bind channel");
+
+ let mut consumer = reader_channel
+ .basic_consume(
+ config.rabbit.queues.acks.into(),
+ "crond-ack-consumer".into(),
+ BasicConsumeOptions::default(),
+ FieldTable::default(),
+ )
+ .await
+ .expect("Failed to create consumer");
+
+ while let Some(delivery) = consumer.next().await {
+ if let Ok(delivery) = delivery {
+ let payload = serde_json::from_slice::(&delivery.data);
+
+ if let Ok(payload) = payload {
+ debug!("Received ack event: {payload:?}");
+
+ if let Err(e) = process_channel_ack(
+ &db,
+ &amqp,
+ payload.user_id,
+ payload.channel_id.unwrap(),
+ &mut redis,
+ )
+ .await
+ {
+ revolt_config::capture_error(&e);
+ _ = delivery.reject(BasicRejectOptions { requeue: false }).await;
+ } else {
+ _ = delivery.ack(BasicAckOptions { multiple: false }).await;
+ }
+ } else {
+ revolt_config::capture_message(
+ format!("Failed to decode ack data: {:?}", delivery.data).as_str(),
+ revolt_config::Level::Error,
+ );
+ }
+ }
+ }
+ Ok(())
+}
+
+#[allow(clippy::disallowed_methods)]
+async fn process_channel_ack(
+ db: &Database,
+ amqp: &AMQP,
+ user: String,
+ channel: String,
+ redis: &mut RedisConnection,
+) -> Result<()> {
+ let message_id: Option = redis
+ .get_del(format!("acker:{user}+{channel}"))
+ .await
+ .to_internal_error()?;
+
+ if let Some(message_id) = message_id {
+ let unread = db.fetch_unread(&user, &channel).await?;
+ let updated = db.acknowledge_message(&channel, &user, &message_id).await?;
+
+ info!("Set new state for ack: {}:{}:{}", channel, user, message_id);
+
+ if let (Some(before), Some(after)) = (unread, updated) {
+ let before_mentions = before.mentions.unwrap_or_default().len();
+ let after_mentions = after.mentions.unwrap_or_default().len();
+
+ if after_mentions < before_mentions {
+ if let Err(err) = amqp
+ .ack_notification_message(user.to_string(), channel.to_string(), message_id)
+ .await
+ {
+ revolt_config::capture_error(&err);
+ }
+ };
+ }
+
+ Ok(())
+ } else {
+ Err(message_id.to_internal_error().expect_err("no err"))
+ }
+}
diff --git a/crates/daemons/crond/src/tasks/file_deletion.rs b/crates/daemons/crond/src/tasks/file_deletion.rs
index 912173b3f..42942bf35 100644
--- a/crates/daemons/crond/src/tasks/file_deletion.rs
+++ b/crates/daemons/crond/src/tasks/file_deletion.rs
@@ -11,22 +11,24 @@ pub async fn task(db: Database) -> Result<()> {
let files = db.fetch_deleted_attachments().await?;
for file in files {
- let count = db
- .count_file_hash_references(file.hash.as_ref().expect("no `hash` present"))
- .await?;
-
- // No other files reference this file on disk anymore
- if count <= 1 {
- let file_hash = db
- .fetch_attachment_hash(file.hash.as_ref().expect("no `hash` present"))
+ if let Some(hash) = &file.hash {
+ let count = db
+ .count_file_hash_references(hash)
.await?;
- // Delete from S3
- delete_from_s3(&file_hash.bucket_id, &file_hash.path).await?;
+ // No other files reference this file on disk anymore
+ if count <= 1 {
+ let file_hash = db
+ .fetch_attachment_hash(hash)
+ .await?;
- // Delete the hash
- db.delete_attachment_hash(&file_hash.id).await?;
- info!("Deleted file hash {}", file_hash.id);
+ // Delete from S3
+ delete_from_s3(&file_hash.bucket_id, &file_hash.path).await?;
+
+ // Delete the hash
+ db.delete_attachment_hash(&file_hash.id).await?;
+ info!("Deleted file hash {}", file_hash.id);
+ }
}
// Delete the file
diff --git a/crates/daemons/crond/src/tasks/mod.rs b/crates/daemons/crond/src/tasks/mod.rs
index 060f55d89..a7dd9040c 100644
--- a/crates/daemons/crond/src/tasks/mod.rs
+++ b/crates/daemons/crond/src/tasks/mod.rs
@@ -1,3 +1,4 @@
+pub mod acks;
pub mod file_deletion;
pub mod prune_dangling_files;
pub mod prune_members;
diff --git a/crates/daemons/pushd/Cargo.toml b/crates/daemons/pushd/Cargo.toml
index 40a280081..9f38020ef 100644
--- a/crates/daemons/pushd/Cargo.toml
+++ b/crates/daemons/pushd/Cargo.toml
@@ -1,42 +1,40 @@
[package]
name = "revolt-pushd"
-version = "0.8.8"
+version = "0.13.7"
edition = "2021"
license = "AGPL-3.0-or-later"
+publish = false
[dependencies]
-revolt-result = { version = "0.8.8", path = "../../core/result" }
-revolt-config = { version = "0.8.8", path = "../../core/config", features = [
- "report-macros",
- "anyhow"
-] }
-revolt-database = { version = "0.8.8", path = "../../core/database" }
-revolt-models = { version = "0.8.8", path = "../../core/models", features = [
- "validator",
-] }
-revolt-presence = { version = "0.8.8", path = "../../core/presence", features = [
- "redis-is-patched",
-] }
+revolt-result = { workspace = true }
+revolt-config = { workspace = true, features = ["report-macros", "anyhow"] }
+revolt-database = { workspace = true }
+revolt-models = { workspace = true, features = ["validator"] }
+revolt-presence = { workspace = true, features = ["redis-is-patched"] }
+revolt-parser = { workspace = true }
-anyhow = { version = "1.0.98" }
+anyhow = { workspace = true }
-amqprs = { version = "1.7.0" }
-fcm_v1 = "0.3.0"
-web-push = "0.10.0"
-isahc = { optional = true, version = "1.7", features = ["json"] }
-revolt_a2 = { version = "0.10", default-features = false, features = ["ring"] }
-tokio = "1.39.2"
-async-trait = "0.1.81"
-ulid = "1.0.0"
+lapin = { workspace = true }
+fcm_v1 = { workspace = true }
+web-push = { workspace = true }
+isahc = { workspace = true, features = ["json"], optional = true }
+revolt_a2 = { workspace = true, features = ["ring"] }
+redis-kiss = { workspace = true }
+tokio = { workspace = true }
+async-trait = { workspace = true }
+ulid = { workspace = true }
-authifier = "1.0.15"
+authifier = { workspace = true }
-log = "0.4.11"
-pretty_env_logger = "0.4.0"
+log = { workspace = true }
+pretty_env_logger = { workspace = true }
+
+regex = { workspace = true }
#serialization
-serde_json = "1"
-revolt_optional_struct = "0.2.0"
-serde = { version = "1", features = ["derive"] }
-iso8601-timestamp = { version = "0.2.10", features = ["serde", "bson"] }
-base64 = "0.22.1"
+serde_json = { workspace = true }
+revolt_optional_struct = { workspace = true }
+serde = { workspace = true }
+iso8601-timestamp = { workspace = true, features = ["serde", "bson"] }
+base64 = { workspace = true }
diff --git a/crates/daemons/pushd/Dockerfile b/crates/daemons/pushd/Dockerfile
index 8123002a0..38bfbc0ee 100644
--- a/crates/daemons/pushd/Dockerfile
+++ b/crates/daemons/pushd/Dockerfile
@@ -1,5 +1,5 @@
# Build Stage
-FROM ghcr.io/revoltchat/base:latest AS builder
+FROM ghcr.io/stoatchat/base:latest AS builder
FROM debian:12 AS debian
# Bundle Stage
diff --git a/crates/daemons/pushd/src/consumers/inbound/ack.rs b/crates/daemons/pushd/src/consumers/inbound/ack.rs
index 77dd5d444..77df8b272 100644
--- a/crates/daemons/pushd/src/consumers/inbound/ack.rs
+++ b/crates/daemons/pushd/src/consumers/inbound/ack.rs
@@ -1,96 +1,69 @@
-use crate::consumers::inbound::internal::*;
-use amqprs::{
- channel::{BasicPublishArguments, Channel},
- connection::Connection,
- consumer::AsyncConsumer,
- BasicProperties, Deliver,
-};
+use std::sync::Arc;
+
+use crate::utils::Consumer;
+use anyhow::Result;
use async_trait::async_trait;
+use lapin::{message::Delivery, Channel, Connection};
use revolt_database::{events::rabbit::*, Database};
+#[derive(Clone)]
+#[allow(unused)]
pub struct AckConsumer {
- #[allow(dead_code)]
db: Database,
authifier_db: authifier::Database,
- conn: Option,
- channel: Option,
-}
-
-impl Channeled for AckConsumer {
- fn get_connection(&self) -> Option<&Connection> {
- if self.conn.is_none() {
- None
- } else {
- Some(self.conn.as_ref().unwrap())
- }
- }
-
- fn get_channel(&self) -> Option<&Channel> {
- if self.channel.is_none() {
- None
- } else {
- Some(self.channel.as_ref().unwrap())
- }
- }
-
- fn set_connection(&mut self, conn: Connection) {
- self.conn = Some(conn);
- }
-
- fn set_channel(&mut self, channel: Channel) {
- self.channel = Some(channel)
- }
+ connection: Arc,
+ channel: Arc,
}
-impl AckConsumer {
- pub fn new(db: Database, authifier_db: authifier::Database) -> AckConsumer {
- AckConsumer {
+#[async_trait]
+impl Consumer for AckConsumer {
+ async fn create(
+ db: Database,
+ authifier_db: authifier::Database,
+ connection: Arc,
+ channel: Arc,
+ ) -> Self {
+ Self {
db,
authifier_db,
- conn: None,
- channel: None,
+ connection,
+ channel,
}
}
-}
-#[allow(unused_variables)]
-#[async_trait]
-impl AsyncConsumer for AckConsumer {
+ fn channel(&self) -> &Arc {
+ &self.channel
+ }
+
/// This consumer processes all acks the platform receives, and sends relevant badge updates to apple platforms.
- async fn consume(
- &mut self,
- channel: &Channel,
- deliver: Deliver,
- basic_properties: BasicProperties,
- content: Vec,
- ) {
- let content = String::from_utf8(content).unwrap();
- let payload: AckPayload = serde_json::from_str(content.as_str()).unwrap();
+ async fn consume(&self, delivery: Delivery) -> Result<()> {
+ let payload: AckPayload = serde_json::from_slice(&delivery.data)?;
// Step 1: fetch unreads and don't continue if there's no unreads
- #[allow(clippy::disallowed_methods)]
- let unreads = self.db.fetch_unread_mentions(&payload.user_id).await;
+ // #[allow(clippy::disallowed_methods)]
debug!("Processing unreads for {:}", &payload.user_id);
- if let Ok(u) = &unreads {
+ let unreads = if let Ok(u) = self.db.fetch_unread_mentions(&payload.user_id).await {
if u.is_empty() {
debug!(
"Discarding unread task (no mentions found) for {:}",
&payload.user_id
);
- return;
- }
+ return Ok(());
+ };
+
+ u
} else {
- return;
- }
+ return Ok(());
+ };
if let Ok(sessions) = self.authifier_db.find_sessions(&payload.user_id).await {
let config = revolt_config::config().await;
// Step 2: find any apple sessions, since we don't need to calculate this for anything else.
// If there's no apple sessions, we can return early
- let apple_sessions: Vec<&authifier::models::Session> = sessions
- .iter()
+ let mut apple_sessions = sessions
+ .into_iter()
.filter(|session| {
if let Some(sub) = &session.subscription {
sub.endpoint == "apn"
@@ -98,19 +71,19 @@ impl AsyncConsumer for AckConsumer {
false
}
})
- .collect();
+ .peekable();
- if apple_sessions.is_empty() {
+ if apple_sessions.peek().is_none() {
debug!(
"Discarding unread task (no apn sessions found) for {:}",
&payload.user_id
);
- return;
+ return Ok(());
}
// Step 3: calculate the actual mention count, since we have to send it out
let mut mention_count = 0;
- for u in &unreads.unwrap() {
+ for u in &unreads {
mention_count += u.mentions.as_ref().unwrap().len()
}
@@ -123,26 +96,22 @@ impl AsyncConsumer for AckConsumer {
token: session.subscription.as_ref().unwrap().auth.clone(),
extras: Default::default(),
};
- let raw_service_payload = serde_json::to_string(&service_payload);
-
- if let Ok(p) = raw_service_payload {
- let args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.apn.queue.as_str(),
- )
- .finish();
-
- log::debug!(
- "Publishing ack to apn session {}",
- session.subscription.as_ref().unwrap().auth
- );
-
- publish_message(self, p.into(), args).await;
- } else {
- log::warn!("Failed to serialize ack badge update payload!");
- revolt_config::capture_error(&raw_service_payload.unwrap_err());
- }
+ let payload = serde_json::to_string(&service_payload)?;
+
+ log::debug!(
+ "Publishing ack to apn session {}",
+ session.subscription.as_ref().unwrap().auth
+ );
+
+ self.publish_message(
+ payload.as_bytes(),
+ &config.pushd.exchange,
+ &config.pushd.apn.queue,
+ )
+ .await?;
}
}
+
+ Ok(())
}
}
diff --git a/crates/daemons/pushd/src/consumers/inbound/dm_call.rs b/crates/daemons/pushd/src/consumers/inbound/dm_call.rs
new file mode 100644
index 000000000..3175d8d8f
--- /dev/null
+++ b/crates/daemons/pushd/src/consumers/inbound/dm_call.rs
@@ -0,0 +1,112 @@
+use std::{collections::HashMap, sync::Arc};
+
+use crate::utils::Consumer;
+use anyhow::Result;
+use async_trait::async_trait;
+use lapin::{message::Delivery, Channel, Connection};
+use log::debug;
+use revolt_database::{events::rabbit::*, Database};
+
+#[derive(Clone)]
+#[allow(unused)]
+pub struct DmCallConsumer {
+ db: Database,
+ authifier_db: authifier::Database,
+ connection: Arc,
+ channel: Arc,
+}
+
+#[async_trait]
+impl Consumer for DmCallConsumer {
+ async fn create(
+ db: Database,
+ authifier_db: authifier::Database,
+ connection: Arc,
+ channel: Arc,
+ ) -> Self {
+ Self {
+ db,
+ authifier_db,
+ connection,
+ channel,
+ }
+ }
+
+ fn channel(&self) -> &Arc {
+ &self.channel
+ }
+
+ /// This consumer handles delegating messages into their respective platform queues.
+ async fn consume(&self, delivery: Delivery) -> Result<()> {
+ let _p: InternalDmCallPayload = serde_json::from_slice(&delivery.data)?;
+ let payload = _p.payload;
+
+ debug!("Received dm call start/stop event");
+
+ let (revolt_database::Channel::DirectMessage { recipients, .. }
+ | revolt_database::Channel::Group { recipients, .. }) =
+ self.db.fetch_channel(&payload.channel_id).await?
+ else {
+ warn!(
+ "Discarding dm call start/stop event for non-dm/group channel {}",
+ payload.channel_id
+ );
+
+ return Ok(());
+ };
+
+ let call_recipients = if let Some(user_recipients) = _p.recipients {
+ user_recipients
+ .into_iter()
+ .filter(|user_id| recipients.contains(user_id) && user_id != &payload.initiator_id)
+ .collect()
+ } else {
+ recipients
+ .into_iter()
+ .filter(|user_id| user_id != &payload.initiator_id)
+ .collect::>()
+ };
+
+ let config = revolt_config::config().await;
+
+ for user_id in call_recipients {
+ if let Ok(sessions) = self.authifier_db.find_sessions(&user_id).await {
+ for session in sessions {
+ if let Some(sub) = session.subscription {
+ let mut sendable = PayloadToService {
+ notification: PayloadKind::DmCallStartEnd(payload.clone()),
+ token: sub.auth,
+ user_id: session.user_id,
+ session_id: session.id,
+ extras: HashMap::new(),
+ };
+
+ let routing_key = match sub.endpoint.as_str() {
+ "apn" => &config.pushd.apn.queue,
+ "fcm" => &config.pushd.fcm.queue,
+ endpoint => {
+ sendable.extras.insert("p256dh".to_string(), sub.p256dh);
+ sendable
+ .extras
+ .insert("endpoint".to_string(), endpoint.to_string());
+
+ &config.pushd.vapid.queue
+ }
+ };
+
+ let payload = serde_json::to_string(&sendable)?;
+
+ self.publish_message(
+ payload.as_bytes(),
+ &config.pushd.exchange,
+ routing_key,
+ )
+ .await?;
+ }
+ }
+ }
+ }
+
+ Ok(())
+ }
+}
diff --git a/crates/daemons/pushd/src/consumers/inbound/fr_accepted.rs b/crates/daemons/pushd/src/consumers/inbound/fr_accepted.rs
index d525138c5..bcfa1c2f0 100644
--- a/crates/daemons/pushd/src/consumers/inbound/fr_accepted.rs
+++ b/crates/daemons/pushd/src/consumers/inbound/fr_accepted.rs
@@ -1,70 +1,44 @@
-use std::collections::HashMap;
+use std::{collections::HashMap, sync::Arc};
-use crate::consumers::inbound::internal::*;
-use amqprs::{
- channel::{BasicPublishArguments, Channel},
- connection::Connection,
- consumer::AsyncConsumer,
- BasicProperties, Deliver,
-};
+use crate::utils::Consumer;
use anyhow::Result;
use async_trait::async_trait;
+use lapin::{message::Delivery, Channel, Connection};
use log::debug;
use revolt_database::{events::rabbit::*, Database};
+#[derive(Clone)]
+#[allow(unused)]
pub struct FRAcceptedConsumer {
- #[allow(dead_code)]
db: Database,
authifier_db: authifier::Database,
- conn: Option,
- channel: Option,
+ connection: Arc,
+ channel: Arc,
}
-impl Channeled for FRAcceptedConsumer {
- fn get_connection(&self) -> Option<&Connection> {
- if self.conn.is_none() {
- None
- } else {
- Some(self.conn.as_ref().unwrap())
- }
- }
-
- fn get_channel(&self) -> Option<&Channel> {
- if self.channel.is_none() {
- None
- } else {
- Some(self.channel.as_ref().unwrap())
- }
- }
-
- fn set_connection(&mut self, conn: Connection) {
- self.conn = Some(conn);
- }
-
- fn set_channel(&mut self, channel: Channel) {
- self.channel = Some(channel)
- }
-}
-
-impl FRAcceptedConsumer {
- pub fn new(db: Database, authifier_db: authifier::Database) -> FRAcceptedConsumer {
- FRAcceptedConsumer {
+#[async_trait]
+impl Consumer for FRAcceptedConsumer {
+ async fn create(
+ db: Database,
+ authifier_db: authifier::Database,
+ connection: Arc,
+ channel: Arc,
+ ) -> Self {
+ Self {
db,
authifier_db,
- conn: None,
- channel: None,
+ connection,
+ channel,
}
}
- async fn consume_event(
- &mut self,
- _channel: &Channel,
- _deliver: Deliver,
- _basic_properties: BasicProperties,
- content: Vec,
- ) -> Result<()> {
- let content = String::from_utf8(content)?;
- let payload: FRAcceptedPayload = serde_json::from_str(content.as_str())?;
+ fn channel(&self) -> &Arc {
+ &self.channel
+ }
+
+ /// This consumer handles delegating messages into their respective platform queues.
+ async fn consume(&self, delivery: Delivery) -> Result<()> {
+ let payload: FRAcceptedPayload = serde_json::from_slice(&delivery.data)?;
debug!("Received FR accept event");
@@ -80,36 +54,23 @@ impl FRAcceptedConsumer {
extras: HashMap::new(),
};
- let args: BasicPublishArguments;
-
- if sub.endpoint == "apn" {
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.apn.queue.as_str(),
- )
- .finish();
- } else if sub.endpoint == "fcm" {
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.fcm.queue.as_str(),
- )
- .finish();
- } else {
- // web push (vapid)
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.vapid.queue.as_str(),
- )
- .finish();
- sendable.extras.insert("p265dh".to_string(), sub.p256dh);
- sendable
- .extras
- .insert("endpoint".to_string(), sub.endpoint.clone());
- }
+ let routing_key = match sub.endpoint.as_str() {
+ "apn" => &config.pushd.apn.queue,
+ "fcm" => &config.pushd.fcm.queue,
+ endpoint => {
+ sendable.extras.insert("p256dh".to_string(), sub.p256dh);
+ sendable
+ .extras
+ .insert("endpoint".to_string(), endpoint.to_string());
+
+ &config.pushd.vapid.queue
+ }
+ };
let payload = serde_json::to_string(&sendable)?;
- publish_message(self, payload.into(), args).await;
+ self.publish_message(payload.as_bytes(), &config.pushd.exchange, routing_key)
+ .await?;
}
}
}
@@ -117,24 +78,3 @@ impl FRAcceptedConsumer {
Ok(())
}
}
-
-#[allow(unused_variables)]
-#[async_trait]
-impl AsyncConsumer for FRAcceptedConsumer {
- /// This consumer handles delegating messages into their respective platform queues.
- async fn consume(
- &mut self,
- channel: &Channel,
- deliver: Deliver,
- basic_properties: BasicProperties,
- content: Vec,
- ) {
- if let Err(err) = self
- .consume_event(channel, deliver, basic_properties, content)
- .await
- {
- revolt_config::capture_anyhow(&err);
- eprintln!("Failed to process friend request accepted event: {err:?}");
- }
- }
-}
diff --git a/crates/daemons/pushd/src/consumers/inbound/fr_received.rs b/crates/daemons/pushd/src/consumers/inbound/fr_received.rs
index f64bc92b8..66fce72df 100644
--- a/crates/daemons/pushd/src/consumers/inbound/fr_received.rs
+++ b/crates/daemons/pushd/src/consumers/inbound/fr_received.rs
@@ -1,70 +1,44 @@
-use std::collections::HashMap;
+use std::{collections::HashMap, sync::Arc};
-use crate::consumers::inbound::internal::*;
-use amqprs::{
- channel::{BasicPublishArguments, Channel},
- connection::Connection,
- consumer::AsyncConsumer,
- BasicProperties, Deliver,
-};
+use crate::utils::Consumer;
use anyhow::Result;
use async_trait::async_trait;
+use lapin::{message::Delivery, Channel, Connection};
use log::debug;
use revolt_database::{events::rabbit::*, Database};
+#[derive(Clone)]
+#[allow(unused)]
pub struct FRReceivedConsumer {
- #[allow(dead_code)]
db: Database,
authifier_db: authifier::Database,
- conn: Option,
- channel: Option,
+ connection: Arc,
+ channel: Arc,
}
-impl Channeled for FRReceivedConsumer {
- fn get_connection(&self) -> Option<&Connection> {
- if self.conn.is_none() {
- None
- } else {
- Some(self.conn.as_ref().unwrap())
- }
- }
-
- fn get_channel(&self) -> Option<&Channel> {
- if self.channel.is_none() {
- None
- } else {
- Some(self.channel.as_ref().unwrap())
- }
- }
-
- fn set_connection(&mut self, conn: Connection) {
- self.conn = Some(conn);
- }
-
- fn set_channel(&mut self, channel: Channel) {
- self.channel = Some(channel)
- }
-}
-
-impl FRReceivedConsumer {
- pub fn new(db: Database, authifier_db: authifier::Database) -> FRReceivedConsumer {
- FRReceivedConsumer {
+#[async_trait]
+impl Consumer for FRReceivedConsumer {
+ async fn create(
+ db: Database,
+ authifier_db: authifier::Database,
+ connection: Arc,
+ channel: Arc,
+ ) -> Self {
+ Self {
db,
authifier_db,
- conn: None,
- channel: None,
+ connection,
+ channel,
}
}
- async fn consume_event(
- &mut self,
- _channel: &Channel,
- _deliver: Deliver,
- _basic_properties: BasicProperties,
- content: Vec,
- ) -> Result<()> {
- let content = String::from_utf8(content)?;
- let payload: FRReceivedPayload = serde_json::from_str(content.as_str())?;
+ fn channel(&self) -> &Arc {
+ &self.channel
+ }
+
+ /// This consumer handles delegating messages into their respective platform queues.
+ async fn consume(&self, delivery: Delivery) -> Result<()> {
+ let payload: FRReceivedPayload = serde_json::from_slice(&delivery.data)?;
debug!("Received FR received event");
@@ -80,36 +54,23 @@ impl FRReceivedConsumer {
extras: HashMap::new(),
};
- let args: BasicPublishArguments;
-
- if sub.endpoint == "apn" {
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.apn.queue.as_str(),
- )
- .finish();
- } else if sub.endpoint == "fcm" {
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.fcm.queue.as_str(),
- )
- .finish();
- } else {
- // web push (vapid)
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.vapid.queue.as_str(),
- )
- .finish();
- sendable.extras.insert("p265dh".to_string(), sub.p256dh);
- sendable
- .extras
- .insert("endpoint".to_string(), sub.endpoint.clone());
- }
+ let routing_key = match sub.endpoint.as_str() {
+ "apn" => &config.pushd.apn.queue,
+ "fcm" => &config.pushd.fcm.queue,
+ endpoint => {
+ sendable.extras.insert("p256dh".to_string(), sub.p256dh);
+ sendable
+ .extras
+ .insert("endpoint".to_string(), endpoint.to_string());
+
+ &config.pushd.vapid.queue
+ }
+ };
let payload = serde_json::to_string(&sendable)?;
- publish_message(self, payload.into(), args).await;
+ self.publish_message(payload.as_bytes(), &config.pushd.exchange, routing_key)
+ .await?;
}
}
}
@@ -117,24 +78,3 @@ impl FRReceivedConsumer {
Ok(())
}
}
-
-#[allow(unused_variables)]
-#[async_trait]
-impl AsyncConsumer for FRReceivedConsumer {
- /// This consumer handles delegating messages into their respective platform queues.
- async fn consume(
- &mut self,
- channel: &Channel,
- deliver: Deliver,
- basic_properties: BasicProperties,
- content: Vec,
- ) {
- if let Err(err) = self
- .consume_event(channel, deliver, basic_properties, content)
- .await
- {
- revolt_config::capture_anyhow(&err);
- eprintln!("Failed to process friend request received event: {err:?}");
- }
- }
-}
diff --git a/crates/daemons/pushd/src/consumers/inbound/generic.rs b/crates/daemons/pushd/src/consumers/inbound/generic.rs
index aa3950d71..f413661d8 100644
--- a/crates/daemons/pushd/src/consumers/inbound/generic.rs
+++ b/crates/daemons/pushd/src/consumers/inbound/generic.rs
@@ -1,70 +1,44 @@
-use std::collections::HashMap;
+use std::{collections::HashMap, sync::Arc};
-use crate::consumers::inbound::internal::*;
-use amqprs::{
- channel::{BasicPublishArguments, Channel},
- connection::Connection,
- consumer::AsyncConsumer,
- BasicProperties, Deliver,
-};
+use crate::utils::Consumer;
use anyhow::Result;
use async_trait::async_trait;
+use lapin::{message::Delivery, Channel, Connection};
use log::debug;
use revolt_database::{events::rabbit::*, Database};
+#[derive(Clone)]
+#[allow(unused)]
pub struct GenericConsumer {
- #[allow(dead_code)]
db: Database,
authifier_db: authifier::Database,
- conn: Option,
- channel: Option,
+ connection: Arc,
+ channel: Arc,
}
-impl Channeled for GenericConsumer {
- fn get_connection(&self) -> Option<&Connection> {
- if self.conn.is_none() {
- None
- } else {
- Some(self.conn.as_ref().unwrap())
- }
- }
-
- fn get_channel(&self) -> Option<&Channel> {
- if self.channel.is_none() {
- None
- } else {
- Some(self.channel.as_ref().unwrap())
- }
- }
-
- fn set_connection(&mut self, conn: Connection) {
- self.conn = Some(conn);
- }
-
- fn set_channel(&mut self, channel: Channel) {
- self.channel = Some(channel)
- }
-}
-
-impl GenericConsumer {
- pub fn new(db: Database, authifier_db: authifier::Database) -> GenericConsumer {
- GenericConsumer {
+#[async_trait]
+impl Consumer for GenericConsumer {
+ async fn create(
+ db: Database,
+ authifier_db: authifier::Database,
+ connection: Arc,
+ channel: Arc,
+ ) -> Self {
+ Self {
db,
authifier_db,
- conn: None,
- channel: None,
+ connection,
+ channel,
}
}
- async fn consume_event(
- &mut self,
- _channel: &Channel,
- _deliver: Deliver,
- _basic_properties: BasicProperties,
- content: Vec,
- ) -> Result<()> {
- let content = String::from_utf8(content)?;
- let payload: MessageSentPayload = serde_json::from_str(content.as_str())?;
+ fn channel(&self) -> &Arc {
+ &self.channel
+ }
+
+ /// This consumer handles delegating messages into their respective platform queues.
+ async fn consume(&self, delivery: Delivery) -> Result<()> {
+ let payload: MessageSentPayload = serde_json::from_slice(&delivery.data)?;
debug!("Received message event on origin");
@@ -86,36 +60,23 @@ impl GenericConsumer {
extras: HashMap::new(),
};
- let args: BasicPublishArguments;
-
- if sub.endpoint == "apn" {
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.apn.queue.as_str(),
- )
- .finish();
- } else if sub.endpoint == "fcm" {
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.fcm.queue.as_str(),
- )
- .finish();
- } else {
- // web push (vapid)
- args = BasicPublishArguments::new(
- config.pushd.exchange.as_str(),
- config.pushd.vapid.queue.as_str(),
- )
- .finish();
- sendable.extras.insert("p265dh".to_string(), sub.p256dh);
- sendable
- .extras
- .insert("endpoint".to_string(), sub.endpoint.clone());
- }
+ let routing_key = match sub.endpoint.as_str() {
+ "apn" => &config.pushd.apn.queue,
+ "fcm" => &config.pushd.fcm.queue,
+ endpoint => {
+ sendable.extras.insert("p256dh".to_string(), sub.p256dh);
+ sendable
+ .extras
+ .insert("endpoint".to_string(), endpoint.to_string());
+
+ &config.pushd.vapid.queue
+ }
+ };
let payload = serde_json::to_string(&sendable)?;
- publish_message(self, payload.into(), args).await;
+ self.publish_message(payload.as_bytes(), &config.pushd.exchange, routing_key)
+ .await?;
}
}
}
@@ -123,24 +84,3 @@ impl GenericConsumer {
Ok(())
}
}
-
-#[allow(unused_variables)]
-#[async_trait]
-impl AsyncConsumer for GenericConsumer {
- /// This consumer handles delegating messages into their respective platform queues.
- async fn consume(
- &mut self,
- channel: &Channel,
- deliver: Deliver,
- basic_properties: BasicProperties,
- content: Vec,
- ) {
- if let Err(err) = self
- .consume_event(channel, deliver, basic_properties, content)
- .await
- {
- revolt_config::capture_anyhow(&err);
- eprintln!("Failed to process generic event: {err:?}");
- }
- }
-}
diff --git a/crates/daemons/pushd/src/consumers/inbound/internal.rs b/crates/daemons/pushd/src/consumers/inbound/internal.rs
deleted file mode 100644
index 387c08b52..000000000
--- a/crates/daemons/pushd/src/consumers/inbound/internal.rs
+++ /dev/null
@@ -1,53 +0,0 @@
-use amqprs::{
- channel::{BasicPublishArguments, Channel},
- connection::{Connection, OpenConnectionArguments},
- BasicProperties,
-};
-use log::{debug, warn};
-
-pub(crate) trait Channeled {
- #[allow(unused)]
- fn get_connection(&self) -> Option<&Connection>;
- fn get_channel(&self) -> Option<&Channel>;
- fn set_connection(&mut self, conn: Connection);
- fn set_channel(&mut self, channel: Channel);
-}
-
-pub(crate) async fn make_channel