* Support path-based S3 storage. Closes#1495
Revert "Remove forcing old path-style URLs with s3. Closes #497"
This reverts commit b2953028cf.
* #1495 Path-style S3 compatibile URLs implemented
https://github.com/owncast/owncast/issues/1495
It gives ability to use S3 compatibile providers that doesn't
support virtual-host-style URLs (i.e. Oracle Cloud Object Storage)
Co-authored-by: Artur Angiel <artur@angiel.ovh>
* add public func to lookup a ChatClient by its clientId
* add facility to send a system message directly to a user
* add clientId field to UserEvent
* implement simple http endpoint to send a message to a user
* let mux handle new directSystemMessageToUser endpoint
* add ClientId to UserEvents across the codebase
* render body of system-message to client
* add clientId to Chat-Message
* add tests showing how url-parsing should work
* add simple rest endpoint helpers for parameter-parsing and easy routing
* use newly added rest-endpoint helper to rout to Client-Messaging controller
* use safe "ReadRestUrlParameter" to parse ClientId
* remove empty HandleFunc in router
* set Header directly to prevent built-in (platform-dependent) canonicalization to kick in
* fix typo in "Parameter" message
* remove debug-logging of HTTP headers in REST-helpers
* convert to uint32 to prevent overruns when converting to wraptype uint later on
* resolve linter-ouchies
* resolve linter potential nil-deref warning
* document the SendSystemMessageToClient endpoint in swaggerdoc
* remove clientId assignment causing potential nil dereference in userDisabledEvent-case
as the clientId isn't relevant here anyway
* make findClientById private, so its not accessible outside of core/chat
* remove redundant string type hint
* Update PR based on linter requirements
Co-authored-by: Raffael Rehberger <raffael@rtrace.io>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>
* First pass at chat user registration and validation
* Disable chat if the user is disabled/blocked or the server hits max connections
* Handle dropping sockets if chat is disabled
* Fix origin in automated chat test
* Work for updated chat moderation
* Chat message markdown rendering and fix tests
* Put /api/chat behind a chat user access token. Closes#1085
* Reject blocked username changes
* More WIP moderation
* Defer configuring chat until we know if it is enabled. Closes#1135
* chat user blocking. Closes#1096
* Add tests around user access for #1096
* Add external integration chat message API + update integration auth middleware to pass along integration name. Closes#1092
* Delete old chat messages from db as to not hold on to excessive data. Closes#1152
* Add schema migration for messages. Closes#1155
* Commit updated API documentation
* Add chat load test
* Shared db mutex and db optimizations
* Simplify past display name handling
* Use a new test db for each test run
* Wire up the external messages actions + add tests for them
* Move access tokens to be actual users
* Run message pruning at launch + fix comparison
* Do not return API users in disabled users response
* Fix incorrect highlighting. Closes#1160
* Consolidate user table statements
* Set the max process connection limit to 70% of maximum
* Fix wrong old display name being returned in name change event
* Delete the old chat server files
* Wire back up the webhooks
* Remove unused
* Invalidate user cache on changes
* Do not send rendered body as RawBody
* Some cleanup
* Standardize names for external API users to ExternalAPIUser
* Do not log token
* Checkout branch when building admin for testing
* Bundle in dev admin for testing
* Some cleanup
* Cleanup js logs
* Cleanup and standardize event names
* Clean up some logging
* Update API spec. Closes#1133
* Commit updated API documentation
* Change paths to be better named
* Commit updated API documentation
* Update admin bundle
* Fix duplicate event name
* Rename scope var
* Update admin bundle
* Move connected clients controller into admin package
* Fix collecting usernames for autocomplete purposes
* No longer generate username when it is empty
* Sort clients and users by timestamp
* Move file to admin controller package
* Swap, so the comments stay correct
Co-authored-by: Jannik <jannik@outlook.com>
* Use explicit type alias
Co-authored-by: Jannik <jannik@outlook.com>
* Remove commented code.
Co-authored-by: Jannik <jannik@outlook.com>
* Cleanup test
* Remove some extra logging
* Add some clarity
* Update dev instance of admin for testing
* Consolidate lines
Co-authored-by: Jannik <jannik@outlook.com>
* Remove commented unused vars
Co-authored-by: Jannik <jannik@outlook.com>
* Until needed do not return IP address with client list
* Fix typo of wrong var
* Typo led to a bad test. Fix typo and fix test.
* Guard against the socket reconnecting on error if previously set to shutdown
* Do not log access tokens
* Return success message on enable/disable user
* Clean up some inactionable error messages. Sent ban message. Sort banned users.
* fix styling for when chat is completely disabled
* Unused
* guard against nil clients
* Update dev admin bundle
* Do not unhide messages when unblocking user just to be safe. Send removal action from the controller
* Add convinience function for getting active connections for a single user
* Lock db on these mutations
* Cleanup force disconnect using GetClientsForUser and capture client reference explicitly
* No longer re-showing banned user messages for safety. Removing this test.
* Remove no longer needed comment
* Tweaks to forbidden username handling.
- Standardize naming to not use "block" but "forbidden" instead.
- Pass array over the wire instead of string.
- Add API test
- Fix default list incorrectly being appended to custom list.
* Logging cleanup
* Update dev admin bundle
* Add an artificial delay in order to visually see message being hidden when testing
* Remove the user cache as it is a premature optimization
* When connected to chat let the user know their current user details to sync the username in the UI
* On connected send current display name back to client.
- Move name change out of chat component.
- Add additional event type constants.
* Fix broken workflow due to typo
* Troubleshoot workflow
* Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181)
* Bump htm from 3.0.4 to 3.1.0 in /build/javascript
Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/developit/htm/releases)
- [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0)
---
updated-dependencies:
- dependency-name: htm
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Run npm run build and update libraries
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>
* Commit updated Javascript packages
* Re-send current user info when a rejected name change takes place
* All socket writes should be through the send chan and not directly
* Seed the random generator
* Add keys and indexes to users table
* a util to generate consistent emoji markup
* console clean up
* mod tidy
* Commit updated API documentation
* Handle the max payload size of a socket message.
- Only close socket if x2 greater than the max size.
- Send the user a message if a message is too large.
- Surface the max size in bytes in the config.
* Update admin bundle
* Force all events to be sent in their own socket message and do not concatinate in a single message
* Update chat embed to register for access token
* Use a different access token for embed chat
* Update the chat message bubble background color to be bolder
* add base tag to open links in new window, closes#1220
* Support text input of :emoji: in chat (#1190)
* Initial implementation of emoji injection
* fix bookkeeping with multiple emoji
* make the emoji lookup case-insensitive
* try another solution for Caretposition
* add title to emojis
minor refactoring
* bind moji injection to InputKeyUp
* simplify the code
replace all found emojis
* inject emoji if the modifer is released earlier
* more efficient emoji tag search
* use json emoji.emoji as url
* use createEmojiMarkup()
* move emojify() to chat.js
* emojify on paste
* cleanup emoji titles in paste
* update inputText in InputKeyup
* mark emoji titles with 2*zwnj
this way paste cleanup will not interfere with text which include zwnj
* emoji should not change the inputText
* Do not show join messages when chat is offline. Closes#1224
- Show stream starting/ending messages in chat.
- When stream starts show everyone the welcome message.
* Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222
* use maxSocketPayloadSize to calculate total bytes of message payload (#1221)
* utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting
* add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count
Co-authored-by: Gabe Kangas <gabek@real-ity.com>
Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Jannik <jannik@outlook.com>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
* Query for installed codecs
* Start modeling out codecs
* Can now specify a codec and get the correct settings returned from the model
* Return codecs in admin/serverconfig
* Start handling transcoding errors and return messages to user
* filter available codecs against a whitelist
* Fix merge
* Codecs are working
* Switching between codecs work
* Add apis for setting a custom video codec
* Cleanup the logging of transcoder errors
* Add v4l codec
* Add fetching v4l
* Add support for per-codec presets
* Use updated nvenc encoding parameters
* Update log message
* Some more codec WIP
* Turn off v4l. It is a mess.
* Try to make the lowest latency level a bit more playable
* Use a human redable display name in console messages
* Turn on transcoder persistent connections
* Add more codec-related user-facing error messages
* Give the initial offline state transcoder an id
* Force a minimum segment count of 3
* Disable qsv for now. set x264 specific params in VariantFlags
* Close body in case
* Ignore vbv underflow message, it is not actionable
* Determine a dynamic gop value based on the length of segments
* Add codec-specific tests
* Cleanup
* Ignore goconst lint warnings in codec file
* Troubleshoot omx
* Add more codec tests
* Remove no longer accurate comment
* Bundle admin from codec branch
* Revert back to old setting
* Cleanup list of codecs a bit
* Remove old references to the encoder preset
* Commit updated API documentation
* Update admin bundle
* Commit updated API documentation
* Add codec setting to api spec
* Commit updated API documentation
Co-authored-by: Owncast <owncast@owncast.online>
* Implement webhook events for external integrations (#574)
* Implement webhook events for external integrations
Reference #556
* move message type to models and remove duplicate
* add json header so content type can be determined
* Pass at migrating webhooks to datastore + management apis (#589)
* Pass at migrating webhooks to datastore + management apis
* Support nil lastUsed timestamps and return back the new webhook on create
* Cleanup from review feedback
* Simplify a bit
Co-authored-by: Aaron Ogle <aaron@geekgonecrazy.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>
* Webhook query cleanup
* Access tokens + Send system message external API (#585)
* New add, get and delete access token APIs
* Create auth token middleware
* Update last_used timestamp when using an access token
* Add auth'ed endpoint for sending system messages
* Cleanup
* Update api spec for new apis
* Commit updated API documentation
* Add auth'ed endpoint for sending user chat messages
* Return access token string
* Commit updated API documentation
* Fix route
* Support nil lastUsed time
* Commit updated Javascript packages
* Remove duplicate function post rebase
* Fix msg id generation
* Update controllers/admin/chat.go
Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>
* Webhook query cleanup
* Add SystemMessageSent to EventType
Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>
* Set webhook as used on completion. Closes#610
* Display webhook errors as errors
* Commit updated API documentation
* Add user joined chat event
* Change integration API paths. Update API spec
* Update development version of admin that supports integration apis
* Commit updated API documentation
* Add automated tests for external integration APIs
* check error
* quiet this test for now
* Route up some additional 3rd party apis. #638
* Commit updated API documentation
* Save username on user joined event
* Add missing scope to valid scopes list
* Add generic chat action event API for 3rd parties. Closes#666
* Commit updated API documentation
* First pass at moving WIP config framework into project for #234
* Only support exported fields in custom types
* Using YP get/set key as a first pass at using the data layer. Fixes + integration.
* Ignore test db
* Start adding getters and setters for config values
* More get/set config work. Starting to populate api with data
* Wire up some config edit endpoints
* More endpoints
* Disable cors middleware
* Add more endpoints and add test to test them
* Remove the in-memory change APIs
* Add endpoint for changing tags
* Add more config endpoints
* Starting to point more things away from config file and to the datastore
* Populate YP with db data
* Create new util method for parsing page body markdown and return it in api
* Verify proposed path to ffmpeg
* For development purposes show the config key in logs
* Move stats values to datastore
* Moving over more values to the datastore
* Move S3 config to datastore
* First pass the config -> db migrator
* Add the start of the video config apis
* It builds pointing everything away from the config
* Tweak ffmpeg path error message
* Backup database every hour. Closes#549
* Config + defaults + migration work for db
* Cleanup logging
* Remove all the old config structs
* Add descriptive info about migration
* Tweak ffmpeg validation logic
* Fix db backup path. backup on db version migration
* Set video and s3 configurations
* Update api spec with new config endpoints
* Add migrator for stats file
* Commit updated API documentation
* Use a dynamic system port for internal HLS writes. Closes#577 (#626)
* Use a dynamic system port for internal HLS writes. Closes#577
* Cleanup
* YP key migration to datastore
* Create a backup directory if needed before migrations
* Remove config test that no longer makes sense. Cleanup.
* Change number types from float32 to float64
* Update automated test suite
* Allow restoring a database backup via command line flags. Closes#549
* Add new hls segment config api
* Commit updated API documentation
* Update apis to require a value container property
* add socialHandles api
* Commit updated API documentation
* Add new latancy level setting to replace segment settings
* Commit updated API documentation
* Fix spelling
* Commit updated API documentation
* hardcode a json api of available social platforms
* Add additional icons
* Return social handles in server config api
* Add socialhandles validation to test
* Move list of hard coded social platforms to an api
* Remove audio only code from transcoder since we do not use it
* Add latency levels api + snapshot of video settings as current broadcast
* Add config/serverurl endpoint
* Return 404 on YP api if disabled
* Surface stream title in YP response
* Add stream title to web ui
* Cleanup log message. Closes#520
* Rename ffmpeg package to transcoder
* Add ws package for testing
* Reduce chat backlog to past 5hrs, max 50. Closes#548
* Fix error formatting
* Add endpoint for resetting yp registration
* Add yp/reset to api spec. return status in response
* Return zero viewer count if stream is offline. Closes#422
* Post-rebase fixes
* Fix merge conflict in openapi file
* Commit updated API documentation
* Standardize controller names
* Support setting the stream key via the command line. Closes#665
* Return social handles with YP data. First half of https://github.com/owncast/owncast-yp/issues/28
* Give the YP package access to server status regardless if enabled or not
* Change delay in automated tests
* Add stream title integration API. For #638
* Commit updated API documentation
* Add storage to the migrator
* Missing returning NSFW value in server config
* Add flag to ignore websocket client. Closes#537
* Add error for parsing broadcaster metadata
* Add support for a cli specified http server port. Closes#674
* Add cpu usage levels and a temporary mapping between it and libx264 presets
* Test for valid url endpoint when saving s3 config
* Re-configure storage on every stream to allow changing storage providers
* After 5 minutes of a stream being stopped clear the stream title
* Hide viewer count once stream goes offline instead of when player stops
* Pull steamTitle from the status that gets updated instead of the config
* Commit updated API documentation
* Optionally show stream title in the header
* Reset stream title when server starts
* Show chat action when stream title is updated
* Allow system messages to come back in persistence
* Split out getting chat history for moderation + fix tests
* Remove server title and standardize on name only
* Commit updated API documentation
* Bump github.com/aws/aws-sdk-go from 1.37.1 to 1.37.2 (#680)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.1 to 1.37.2.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.1...v1.37.2)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add video variant and stream latency config file migrator
* Remove mostly unused disable upgrade check bool
* Commit updated API documentation
* Allow bundling the admin from the 0.0.6 branch
* Fix saving port numbers
* Use name instead of old title on window focus
* Work on latency levels. Fix test to use levels. Clean up transcoder to only reference levels
* Another place where title -> name
* Fix test
* Bump github.com/aws/aws-sdk-go from 1.37.2 to 1.37.3 (#690)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.2 to 1.37.3.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.2...v1.37.3)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Update dependabot config
* Bump github.com/aws/aws-sdk-go from 1.37.3 to 1.37.5 (#693)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.3 to 1.37.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.3...v1.37.5)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript (#694)
* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript
Bumps [video.js](https://github.com/videojs/video.js) from 7.10.2 to 7.11.4.
- [Release notes](https://github.com/videojs/video.js/releases)
- [Changelog](https://github.com/videojs/video.js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/videojs/video.js/compare/v7.10.2...v7.11.4)
Signed-off-by: dependabot[bot] <support@github.com>
* Commit updated Javascript packages
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>
* Make the latency migrator dynamic so I can tweak values easier
* Split out fetching ffmpeg path from validating the path so it can be changed in the admin
* Some commenting and linter cleanup
* Validate the path for a logo change and throw an error if it does not exist
* Logo change requests have to be a real file now
* Cleanup, making linter happy
* Format javascript on push
* Only format js in master
* Tweak latency level values
* Remove unused config file examples
* Fix thumbnail generation after messing with the ffmpeg path getter
* Reduce how often we report high hardware utilization warnings
* Bundle the 0.0.6 branch version of the admin
* Return validated ffmpeg path in admin server config
* Change the logo to be stored in the data directory instead of webroot
* Bump postcss from 8.2.4 to 8.2.5 in /build/javascript (#702)
Bumps [postcss](https://github.com/postcss/postcss) from 8.2.4 to 8.2.5.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.2.4...8.2.5)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Default config file no longer used
* don't show stream title when offline
addresses https://github.com/owncast/owncast/issues/677
* Remove auto-clearing stream title. #677
* webroot -> data when using logo as thumbnail
* Do not list websocket/access token create/delete as integration APIs
* Commit updated API documentation
* Bundle updated admin
* Remove pointing to the 0.0.6 admin branch
* Linter cleanup
* Linter cleanup
* Add donations and follow links to show up under social handles
* Prettified Code!
* More linter cleanup
* Update admin bundle
* Remove use of platforms.js and return icons with social handles. Closes#732
* Update admin bundle
* Support custom config path for use in migration
* Remove unused platform-logos.gif
* Reduce log level of message
* Remove unused logo files in static dir
* Handle dev vs. release build info
* Restore logo.png for initial thumbnail
* Cleanup some files from the build process that are not needed
* Fix incorrect build-time injection var
* Fix missing file getting copied to the build
* Remove console directory message.
* Update admin bundle
* Fix comment
* Report storage setup error
* add some value set error checking
* Use validated dynamic ffmpeg path for animated gif preview
* Make chat message links be white so they don't hide in the bg. Closes#599
* Restore conditional that was accidentally removed
Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: nebunez <uoj2y7wak869@opayq.net>
Co-authored-by: gabek <gabek@users.noreply.github.com>
* update message viz in db
* create admin endpoint to update message visibility
* convert UpdateMessageVisibility api to take in an array of IDs to change visibility on instead
* Support requesting filtered or unfiltered chat messages
* Handle UPDATE chat events on front and backend for toggling messages
* Return entire message with UPDATE events
* Remove the UPDATE message type
* Revert "Remove the UPDATE message type"
This reverts commit 3a83df3d492f7ecf2bab65e845aa2b0365d3a7f6.
* update -> visibility update
* completely remove messages when they turn hidden on VISIBILITY-UPDATEs, and insert them if they turn visible
* Explicitly set visibility
* Fix multi-id sql updates
* increate scroll buffer a bit so chat scrolls when new large messages come in
* Add automated test around chat moderation
* Add new chat admin APIs to api spec
* Commit updated API documentation
Co-authored-by: Gabe Kangas <gabek@real-ity.com>
Co-authored-by: Owncast <owncast@owncast.online>
* Simplify Logo requirement from users.
- Only require 1 logo file, instead of a `small` and `large` one. Just require `logo`.
- Update frontend sso that primary header logo will ALWAYS be owncast logo.
- User's logo will remain in "user content" area.
* Commit updated API documentation
Co-authored-by: Owncast <owncast@owncast.online>
* Render and sanitize chat messages server-side. Closes#235
* Render content.md server-side and return it in the client config
* Remove showdown from web project
* Update api spec
* Move example user content file
Gabe Kangas
Artur Angiel
Ruffy
Mirco T
gingervitis
Jannik Volkland
John Hobbs