Compare commits

..

No commits in common. 'develop' and 'gek/test-caches' have entirely different histories.

  1. 19
      .design/DESIGN.md
  2. 23
      .github/PULL_REQUEST_TEMPLATE/atomic-refactor-pull-request-for-issue-2119.md
  3. 4
      .github/PULL_REQUEST_TEMPLATE/standard-pull-request.md
  4. 1
      .github/codeql/go.yml
  5. 4
      .github/codeql/javascript.yml
  6. 2
      .github/workflows/actions-lint.yml
  7. 2
      .github/workflows/auto-comment-on-label.yaml
  8. 4
      .github/workflows/automated-end-to-end-api.yaml
  9. 21
      .github/workflows/browser-testing.yml
  10. 4
      .github/workflows/build-storybook.yml
  11. 44
      .github/workflows/bundle-web.yml
  12. 2
      .github/workflows/chromatic.yml
  13. 9
      .github/workflows/codeql-analysis.yml
  14. 2
      .github/workflows/container-lint.yml
  15. 4
      .github/workflows/container.yaml
  16. 2
      .github/workflows/generate-api-documentation.yaml
  17. 8
      .github/workflows/go-lint.yml
  18. 21
      .github/workflows/hls-tests.yml
  19. 185
      .github/workflows/javascript-format-build.yml
  20. 69
      .github/workflows/javascript-formatting.yml
  21. 8
      .github/workflows/screenshots.yml
  22. 4
      .github/workflows/shellcheck.yml
  23. 33
      .github/workflows/test-webapp-build.yaml
  24. 28
      .github/workflows/test.yaml
  25. 4
      .gitignore
  26. 4
      .golangci.yml
  27. 28
      .vscode/settings.json
  28. 2
      Dockerfile
  29. 29
      Earthfile
  30. 2
      LICENSE
  31. 18
      README.md
  32. 16
      activitypub/apmodels/actor.go
  33. 48
      activitypub/apmodels/utils.go
  34. 14
      activitypub/apmodels/webfinger.go
  35. 8
      activitypub/controllers/nodeinfo.go
  36. 8
      activitypub/inbox/like.go
  37. 10
      activitypub/inbox/workerpool.go
  38. 2
      activitypub/outbox/outbox.go
  39. 107
      activitypub/persistence/followers_test.go
  40. 4
      activitypub/resolvers/resolve.go
  41. 17
      activitypub/webfinger/webfinger.go
  42. 9
      activitypub/workerpool/outbound.go
  43. 2
      auth/fediverse/fediverse.go
  44. 26
      auth/indieauth/client.go
  45. 10
      auth/indieauth/request.go
  46. 4
      auth/indieauth/server.go
  47. 18
      build/web/bundleWeb.sh
  48. 2
      config/constants.go
  49. 37
      config/defaults.go
  50. 4
      config/verifyInstall.go
  51. 82
      contrib/owncast_for_windows.md
  52. 12
      controllers/admin/chat.go
  53. 65
      controllers/admin/config.go
  54. 1
      controllers/admin/externalAPIUsers.go
  55. 2
      controllers/admin/logs.go
  56. 54
      controllers/admin/serverConfig.go
  57. 8
      controllers/admin/status.go
  58. 3
      controllers/admin/update.go
  59. 1
      controllers/chat.go
  60. 27
      controllers/config.go
  61. 5
      controllers/controllers.go
  62. 3
      controllers/emoji.go
  63. 98
      controllers/index.go
  64. 6
      controllers/moderation/moderation.go
  65. 2
      controllers/pagination.go
  66. 28
      controllers/robots.go
  67. 4
      controllers/status.go
  68. 6
      controllers/video.go
  69. 28
      core/chat/chatclient.go
  70. 1
      core/chat/concurrentConnections_freebsd.go
  71. 7
      core/chat/events.go
  72. 2
      core/chat/events/connectedClientInfo.go
  73. 135
      core/chat/events/events.go
  74. 6
      core/chat/events/eventtype.go
  75. 17
      core/chat/events/userPartEvent.go
  76. 25
      core/chat/messageRendering_test.go
  77. 35
      core/chat/persistence.go
  78. 79
      core/chat/server.go
  79. 8
      core/core.go
  80. 138
      core/data/config.go
  81. 2
      core/data/configEntry.go
  82. 6
      core/data/data_test.go
  83. 14
      core/data/datastoreMigrations.go
  84. 84
      core/data/emoji.go
  85. 4
      core/data/migrations.go
  86. 10
      core/data/types.go
  87. 16
      core/rtmp/rtmp.go
  88. 95
      core/storageproviders/local.go
  89. 43
      core/storageproviders/rewriteLocalPlaylist.go
  90. 175
      core/storageproviders/s3Storage.go
  91. 9
      core/streamState.go
  92. 42
      core/transcoder/codecs.go
  93. 3
      core/transcoder/fileWriterReceiverService.go
  94. 78
      core/transcoder/hlsFilesystemCleanup.go
  95. 8
      core/transcoder/thumbnailGenerator.go
  96. 50
      core/transcoder/transcoder.go
  97. 2
      core/transcoder/transcoder_nvenc_test.go
  98. 2
      core/transcoder/transcoder_omx_test.go
  99. 2
      core/transcoder/transcoder_vaapi_test.go
  100. 2
      core/transcoder/transcoder_videotoolbox_test.go
  101. Some files were not shown because too many files have changed in this diff Show More

19
.design/DESIGN.md

@ -27,32 +27,21 @@ Read the detailed [product definition](https://github.com/owncast/owncast/blob/d @@ -27,32 +27,21 @@ Read the detailed [product definition](https://github.com/owncast/owncast/blob/d
Owncast is a is a live streaming and chat server targeted to anybody who has live streaming needs. This means anything from corporate events, government meetings, game streams, concerts, TV stations, and more.
## 🧑‍🎨 Product design opportunities
Owncast is a constantly moving project with features both old and new. This allows for design contributions to be both big or small.
You may not know how much time you can dedicate to the project, or if you'll be able to see something through to the end, so be honest about that. Take on projects that you'll be able to see completed.
- So maybe start small by finding rough edges and improvements to existing features without requiring complete rewrites. As a small project the bandwidth for rebuilding existing designs is limited, but tweaks are appreciated. This is especially great if you don't know how much time or energy you'll be able to provide the project. If you think you have a week to help, but might not be around in a month small projects are better.
- If you think you'll be around longer term, learn about future new features and start thinking about the design challenges of those so we can build them your feedback and design contributions in mind. See your designs put in the world through brand new functionality!
- Not everything has to be a a feature. Think big picture. What can we start doing now to put the project in a better place six months from now, or a year?
## 💅 Design relevant materials
A collection of design relevant information and materials can be found under the "style" section of "Storybook" here:
http://owncast.online/components
Here is a list of design relevant information and materials:
### Fonts
https://owncast.online/components/?path=%2Fdocs%2Fowncast-styles-typography--page
http://owncast.online/components/?path=/story/owncast-style-guide-typography--page
Body text: Inter
Body text: OpenSans
Display/Header text: Poppins
### Colors
https://owncast.online/components/?path=%2Fdocs%2Fowncast-styles-colors-components--page
http://owncast.online/components/?path=/story/owncast-style-guide-default-theme--page
### Design Files, Screenshots, etc

23
.github/PULL_REQUEST_TEMPLATE/atomic-refactor-pull-request-for-issue-2119.md

@ -0,0 +1,23 @@ @@ -0,0 +1,23 @@
<!-- this template is for changes relating to #2119. You might want to use the standard template. -->
# Description
<!-- do not remove -->
This PR is for updating/adding a component following the atomic design pattern set out in #2119.
<!-- mention the component you changed, and describe any design choices if necessary -->
---
Extra Info
<!-- fill these in -->
- Component name in [kanban board](https://collab.owncast.tv/kanban/#/2/kanban/edit/omLI2N+LcnP+elmdT7qW9GHD/): `________`
Checklist:
- [] The component follows the [design guide](../../web/components/_COMPONENT_HOW_TO.md).
- [] Moved the component to the correct `atoms` / `molecules` / `organisms` / `templates` directory.
- [] Added an explanation to this PR for any major changes you made.
- [] Replaced any [`defaultProps`](https://www.reactjstutorials.com/react-basics/17/react-default-props) with default args.
- [] Added a (short) JSDoc description to the component.
- [] Removed the component's Storybook description text with if it's not needed.

4
.github/PULL_REQUEST_TEMPLATE/standard-pull-request.md

@ -1,8 +1,6 @@ @@ -1,8 +1,6 @@
Please include a summary of the change and which issue number is fixed, including relevant motivation and context. Feel free to mark this as a Draft or WIP and write up some details later.
If there is no issue filed for this particular change it's highly recommended you file one. While creating this PR means you probably already did the work, in the future make sure an issue is filed beforehand so changes, fixes and features can be discussed ahead of time.
# Description
Fixes # (issue)
@ -16,4 +14,4 @@ Some things you might want to mention: @@ -16,4 +14,4 @@ Some things you might want to mention:
3. If you're fixing something, what was wrong? How should we stop from having this issue happen again?
4. If this is a new feature or addition to functionality, why should it be added? What are the use cases? Who was asking for this functionality?
If this is an unsolicited change or have no issue associated please do your best to detail the motivations behind this PR, and think about filing an issue to discuss changes ahead of time in the future.
If this is an unsolicited change or have no issue associated please do your best to detail the motivations behind this PR.

1
.github/codeql/go.yml

@ -1 +0,0 @@ @@ -1 +0,0 @@
name: Go config

4
.github/codeql/javascript.yml

@ -1,4 +0,0 @@ @@ -1,4 +0,0 @@
name: Javascript config
paths-ignore:
- static/web

2
.github/workflows/actions-lint.yml

@ -13,7 +13,7 @@ jobs: @@ -13,7 +13,7 @@ jobs:
name: GitHub actions
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- uses: docker://rhysd/actionlint:latest
with:

2
.github/workflows/auto-comment-on-label.yaml

@ -11,7 +11,7 @@ jobs: @@ -11,7 +11,7 @@ jobs:
issues: write
steps:
- name: Add comment
uses: peter-evans/create-or-update-comment@0f44b017d10caeea6a4c1b410ba0521ad8a02815
uses: peter-evans/create-or-update-comment@300bfadd23e5f84577e47387d4acebeeb87fea2d
with:
issue-number: ${{ github.event.issue.number }}
body: |

4
.github/workflows/automated-end-to-end-api.yaml

@ -27,12 +27,12 @@ jobs: @@ -27,12 +27,12 @@ jobs:
- name: Set up QEMU
id: qemu
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v2
with:
image: tonistiigi/binfmt:latest
platforms: all
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Run API tests
uses: nick-fields/retry@v2

21
.github/workflows/browser-testing.yml

@ -9,7 +9,6 @@ on: @@ -9,7 +9,6 @@ on:
paths:
- 'web/**'
- 'test/automated/browser/**'
jobs:
cypress-run:
runs-on: ubuntu-latest
@ -20,14 +19,18 @@ jobs: @@ -20,14 +19,18 @@ jobs:
concurrent_skipping: 'same_content_newer'
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v3
- uses: actions/setup-node@v4
- uses: actions/setup-go@v3
with:
node-version: 18.9.0
go-version: '1.18.8'
cache: true
- name: Install Google Chrome
run: sudo apt-get install google-chrome-stable
- name: Cache node modules
uses: actions/cache@v3
uses: actions/cache@v2
env:
cache-name: cache-node-modules-browser-tests
with:
@ -38,14 +41,6 @@ jobs: @@ -38,14 +41,6 @@ jobs:
${{ runner.os }}-build-
${{ runner.os }}-
- uses: actions/setup-go@v5
with:
go-version: '1.21'
cache: true
- name: Install Google Chrome
run: sudo apt-get update && sudo apt-get install google-chrome-stable
- name: Run Browser tests
uses: nick-fields/retry@v2
with:

4
.github/workflows/build-storybook.yml

@ -12,10 +12,10 @@ jobs: @@ -12,10 +12,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v3
- name: Cache node modules
uses: actions/cache@v3
uses: actions/cache@v2
env:
cache-name: cache-node-modules-bundle-web-app
with:

44
.github/workflows/bundle-web.yml

@ -0,0 +1,44 @@ @@ -0,0 +1,44 @@
name: Build and bundle web app into Owncast
on:
push:
branches:
- develop
paths:
- 'web/**'
- '!**.md'
jobs:
bundle:
runs-on: ubuntu-latest
if: github.repository == 'owncast/owncast'
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
- name: Cache node modules
uses: actions/cache@v2
env:
cache-name: cache-node-modules-bundle-web-app
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('web/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Bundle web app (next.js build)
uses: actions/checkout@v3
- run: build/web/bundleWeb.sh
- name: Commit changes
uses: EndBug/add-and-commit@v9
with:
pull: --rebase --autostash
message: 'Bundle embedded web app'
add: 'static/web'
author_name: Owncast
author_email: owncast@owncast.online

2
.github/workflows/chromatic.yml

@ -29,7 +29,7 @@ jobs: @@ -29,7 +29,7 @@ jobs:
concurrent_skipping: 'same_content_newer'
- name: Check out code
if: ${{ github.actor != 'renovate[bot]' && github.actor != 'renovate' }}
uses: actions/checkout@v4
uses: actions/checkout@v3
with:
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}

9
.github/workflows/codeql-analysis.yml

@ -37,14 +37,13 @@ jobs: @@ -37,14 +37,13 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/${{ matrix.language }}.yml
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
@ -53,7 +52,7 @@ jobs: @@ -53,7 +52,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3
uses: github/codeql-action/autobuild@v2
# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@ -67,4 +66,4 @@ jobs: @@ -67,4 +66,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@v2

2
.github/workflows/container-lint.yml

@ -19,7 +19,7 @@ jobs: @@ -19,7 +19,7 @@ jobs:
container:
image: aquasec/trivy
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Check critical issues
run: trivy config --exit-code 1 --severity "HIGH,CRITICAL" ./Dockerfile

4
.github/workflows/container.yaml

@ -32,13 +32,13 @@ jobs: @@ -32,13 +32,13 @@ jobs:
run: echo "${{ secrets.GH_CR_PAT }}" | docker login https://ghcr.io -u ${{ github.actor }} --password-stdin
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v2
with:
image: tonistiigi/binfmt:latest
platforms: all
- name: Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@v3
with:
fetch-depth: 0

2
.github/workflows/generate-api-documentation.yaml

@ -10,7 +10,7 @@ jobs: @@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@v3
- name: Run redoc on openapi.yaml
run: |

8
.github/workflows/go-lint.yml

@ -22,15 +22,15 @@ jobs: @@ -22,15 +22,15 @@ jobs:
with:
concurrent_skipping: 'same_content_newer'
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: actions/setup-go@v5
- uses: actions/setup-go@v3
with:
go-version: '1.21'
go-version: '1.18.8'
cache: true
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:

21
.github/workflows/hls-tests.yml

@ -1,4 +1,4 @@ @@ -1,4 +1,4 @@
name: HLS tests
name: Automated HLS tests
on:
push:
@ -16,7 +16,7 @@ env: @@ -16,7 +16,7 @@ env:
S3_SECRET: ${{ secrets.S3_SECRET }}
jobs:
tests:
api:
runs-on: ubuntu-latest
steps:
@ -24,14 +24,14 @@ jobs: @@ -24,14 +24,14 @@ jobs:
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
with:
go-version: '1.21'
go-version: '1.18.8'
cache: true
- name: Cache node modules
uses: actions/cache@v3
uses: actions/cache@v2
env:
cache-name: cache-node-modules-hls-tests
with:
@ -42,16 +42,9 @@ jobs: @@ -42,16 +42,9 @@ jobs:
${{ runner.os }}-build-
${{ runner.os }}-
- name: Local stroage
- name: Run HLS tests
uses: nick-fields/retry@v2
with:
timeout_minutes: 10
max_attempts: 3
command: cd test/automated/hls && ./run.sh
- name: S3 storage
uses: nick-fields/retry@v2
with:
timeout_minutes: 10
max_attempts: 3
command: cd test/automated/hls && ./run-s3.sh

185
.github/workflows/javascript-format-build.yml

@ -1,185 +0,0 @@ @@ -1,185 +0,0 @@
name: Javascript
# This action works with pull requests and pushes
on:
push:
paths:
- web/**
- '!**.md'
pull_request:
paths:
- web/**
- '!**.md'
jobs:
formatting:
name: Code formatting
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./web
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Checkout
uses: actions/checkout@v4
with:
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
persist-credentials: true
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v41
with:
path: 'web'
files_ignore: |
static/**
web/next.config.js
files_yaml: |
src:
- '**/*.{js,ts,tsx,jsx,css,md}'
- name: Cache node modules
uses: actions/cache@v3
env:
cache-name: cache-node-modules-bundle-web-app
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('web/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Install Dependencies
run: npm install
- name: Lint
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: npx eslint --fix ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
- name: Prettier
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: npx prettier --write ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
- name: Commit changes
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
uses: EndBug/add-and-commit@v9
with:
author_name: Owncast
author_email: owncast@owncast.online
message: 'Javascript formatting autofixes'
add: ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
pull: '--rebase --autostash'
unused-code:
name: Test for unused code
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./web
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Checkout
uses: actions/checkout@v4
with:
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Cache node modules
uses: actions/cache@v3
env:
cache-name: cache-node-modules-bundle-web-app
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('web/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Install Dependencies
run: npm install
- name: Check for unused JS code and dependencies
run: npx knip --include dependencies,files,exports
# After any formatting and linting is complete we can run the build
# and bundle step. This both will verify that the build is successful as
# well as commiting the updated static files into the repository for use.
web-bundle:
name: Build and bundle web project
runs-on: ubuntu-latest
if: github.repository == 'owncast/owncast'
needs: [formatting, unused-code]
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Cache node modules
uses: actions/cache@v3
env:
cache-name: cache-node-modules-bundle-web-app
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('web/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Checkout
uses: actions/checkout@v4
with:
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Bundle web app (next.js build)
run: build/web/bundleWeb.sh
- name: Rebase
if: ${{ github.ref == 'refs/heads/develop' }}
run: |
git add static/web
git pull --rebase --autostash
# Only commit built web project files on develop.
- name: Commit changes
if: ${{ github.ref == 'refs/heads/develop' }}
uses: EndBug/add-and-commit@v9
with:
message: 'Bundle embedded web app'
add: 'static/web'
author_name: Owncast
author_email: owncast@owncast.online
- name: Push changes
if: ${{ github.ref == 'refs/heads/develop' }}
run: |
git pull --rebase --autostash
git push

69
.github/workflows/javascript-formatting.yml

@ -0,0 +1,69 @@ @@ -0,0 +1,69 @@
name: Lint
# This action works with pull requests and pushes
on:
push:
paths:
- web/**
pull_request_target:
paths:
- web/**
jobs:
prettier:
name: Javascript prettier
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./web
if: ${{ github.actor != 'dependabot[bot]' }}
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
- name: Checkout
uses: actions/checkout@v3
with:
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Prettify code
uses: creyD/prettier_action@v4.2
with:
# This part is also where you can pass other options, for example:
prettier_options: --write **/*.{js,ts,jsx,tsx,css,md}
only_changed: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
linter:
name: Javascript linter
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./web
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
- name: Checkout
uses: actions/checkout@v3
with:
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Install Dependencies
run: npm install
- name: Lint
run: npm run lint

8
.github/workflows/screenshots.yml

@ -14,14 +14,14 @@ jobs: @@ -14,14 +14,14 @@ jobs:
Screenshots:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
with:
go-version: '1.21'
go-version: '1.18.8'
cache: true
- name: Cache node modules
uses: actions/cache@v3
uses: actions/cache@v2
env:
cache-name: cache-node-modules-screenshots
with:

4
.github/workflows/shellcheck.yml

@ -18,9 +18,9 @@ jobs: @@ -18,9 +18,9 @@ jobs:
env:
LANG: C.UTF-8
container:
image: docker.io/ubuntu:24.04
image: docker.io/ubuntu:23.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Install shellcheck
run: apt update && apt install -y shellcheck bash && shellcheck --version

33
.github/workflows/javascript-tests.yml → .github/workflows/test-webapp-build.yaml

@ -1,16 +1,22 @@ @@ -1,16 +1,22 @@
name: Javascript Tests
name: Webapp Test Build
# This action works with pull requests and pushes
on:
push:
paths:
- 'web/**'
- web/**
pull_request:
paths:
- 'web/**'
- web/**
jobs:
jest-run:
build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./web
name: Build webapp
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
@ -18,16 +24,17 @@ jobs: @@ -18,16 +24,17 @@ jobs:
concurrent_skipping: 'same_content_newer'
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-node@v4
uses: actions/checkout@v3
with:
node-version: 18.9.0
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Cache node modules
uses: actions/cache@v3
uses: actions/cache@v2
env:
cache-name: cache-node-modules-javascript-tests
cache-name: cache-node-modules-bundle-web-app
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('web/package-lock.json') }}
@ -37,9 +44,7 @@ jobs: @@ -37,9 +44,7 @@ jobs:
${{ runner.os }}-
- name: Install Dependencies
working-directory: ./web
run: npm install
- name: Run tests
working-directory: ./web
run: npm test
- name: Build
run: npm run build

28
.github/workflows/go-tests.yaml → .github/workflows/test.yaml

@ -12,23 +12,14 @@ jobs: @@ -12,23 +12,14 @@ jobs:
test:
strategy:
matrix:
go-version: [1.20.x, 1.21.x]
go-version: [1.18.x, 1.19.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/cache@v3
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: go-test-${{ github.sha }}
restore-keys: |
go-test-
- uses: actions/checkout@v3
- name: Install go
uses: actions/setup-go@v5
uses: actions/setup-go@v3
with:
go-version: '^1'
cache: true
@ -47,19 +38,10 @@ jobs: @@ -47,19 +38,10 @@ jobs:
version: 6.8
steps:
- uses: actions/checkout@v4
- uses: actions/cache@v3
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: go-test-${{ github.sha }}
restore-keys: |
go-test-
- uses: actions/checkout@v3
- name: Install go
uses: actions/setup-go@v5
uses: actions/setup-go@v3
with:
go-version: '^1'
cache: true

4
.gitignore vendored

@ -42,7 +42,3 @@ test/automated/browser/screenshots @@ -42,7 +42,3 @@ test/automated/browser/screenshots
lefthook.yml
test/automated/browser/cypress/screenshots
test/automated/browser/cypress/videos
web/style-definitions/build/
web/public/sw.js
web/public/workbox-*.js

4
.golangci.yml

@ -5,7 +5,7 @@ run: @@ -5,7 +5,7 @@ run:
# Define the Go version limit.
# Mainly related to generics support in go1.18.
# Default: use Go version from the go.mod file, fallback on the env var `GOVERSION`, fallback on 1.18
go: '1.21'
go: '1.18'
issues:
# The linter has a default list of ignorable errors. Turning this on will enable that list.
@ -69,7 +69,7 @@ linters-settings: @@ -69,7 +69,7 @@ linters-settings:
gosimple:
# Select the Go version to target. The default is '1.13'.
go: '1.21'
go: '1.18'
# https://staticcheck.io/docs/options#checks
checks: ['all']

28
.vscode/settings.json vendored

@ -0,0 +1,28 @@ @@ -0,0 +1,28 @@
{
"cSpell.words": [
"Debugln",
"Errorln",
"Fediverse",
"Ffmpeg",
"ffmpegpath",
"ffmpg",
"geoip",
"gosec",
"mattn",
"Mbps",
"nolint",
"Owncast",
"ppid",
"preact",
"RTMP",
"rtmpserverport",
"sqlite",
"Tracef",
"Traceln",
"upgrader",
"Upgrader",
"videojs",
"Warnf",
"Warnln"
]
}

2
Dockerfile

@ -22,7 +22,7 @@ ENV NAME=${NAME} @@ -22,7 +22,7 @@ ENV NAME=${NAME}
RUN CGO_ENABLED=1 GOOS=linux go build -a -installsuffix cgo -ldflags "-extldflags \"-static\" -s -w -X github.com/owncast/owncast/config.GitCommit=$GIT_COMMIT -X github.com/owncast/owncast/config.VersionNumber=$VERSION -X github.com/owncast/owncast/config.BuildPlatform=$NAME" -o owncast .
# Create the image by copying the result of the build into a new alpine image
FROM alpine:3.19.0
FROM alpine:3.17.2
RUN apk update && apk add --no-cache ffmpeg ffmpeg-libs ca-certificates && update-ca-certificates
RUN addgroup -g 101 -S owncast && adduser -u 101 -S owncast -G owncast

29
Earthfile

@ -6,10 +6,10 @@ ARG version=develop @@ -6,10 +6,10 @@ ARG version=develop
WORKDIR /build
build-all:
BUILD --platform=linux/amd64 --platform=linux/386 --platform=linux/arm64 --platform=linux/arm/v7 --platform=darwin/amd64 --platform=darwin/arm64 +build
BUILD --platform=linux/amd64 --platform=linux/386 --platform=linux/arm64 --platform=linux/arm/v7 --platform=darwin/amd64 +build
package-all:
BUILD --platform=linux/amd64 --platform=linux/386 --platform=linux/arm64 --platform=linux/arm/v7 --platform=darwin/amd64 --platform=darwin/arm64 +package
BUILD --platform=linux/amd64 --platform=linux/386 --platform=linux/arm64 --platform=linux/arm/v7 --platform=darwin/amd64 +package
docker-all:
BUILD --platform=linux/amd64 --platform=linux/386 --platform=linux/arm64 --platform=linux/arm/v7 +docker
@ -17,7 +17,7 @@ docker-all: @@ -17,7 +17,7 @@ docker-all:
crosscompiler:
# This image is missing a few platforms, so we'll add them locally
FROM --platform=linux/amd64 bdwyertech/go-crosscompile
RUN apk add --update --no-cache tar gzip upx >> /dev/null
RUN apk add --update --no-cache tar gzip >> /dev/null
RUN curl -sfL "https://owncast-infra.nyc3.cdn.digitaloceanspaces.com/build/armv7l-linux-musleabihf-cross.tgz" | tar zxf - -C /usr/ --strip-components=1
RUN curl -sfL "https://owncast-infra.nyc3.cdn.digitaloceanspaces.com/build/i686-linux-musl-cross.tgz" | tar zxf - -C /usr/ --strip-components=1
RUN curl -sfL "https://owncast-infra.nyc3.cdn.digitaloceanspaces.com/build/x86_64-linux-musl-cross.tgz" | tar zxf - -C /usr/ --strip-components=1
@ -36,6 +36,7 @@ build: @@ -36,6 +36,7 @@ build:
FROM --platform=linux/amd64 +code
RUN echo $EARTHLY_GIT_HASH
RUN echo "Finding CC configuration for $TARGETPLATFORM"
IF [ "$TARGETPLATFORM" = "linux/amd64" ]
ARG NAME=linux-64bit
@ -58,10 +59,6 @@ build: @@ -58,10 +59,6 @@ build:
ARG NAME=macOS-64bit
ARG CC=o64-clang
ARG CXX=o64-clang++
ELSE IF [ "$TARGETPLATFORM" = "darwin/arm64" ]
ARG NAME=macOS-arm64
ARG CC=o64-clang
ARG CXX=o64-clang++
ELSE
RUN echo "Failed to find CC configuration for $TARGETPLATFORM"
ARG --required CC
@ -79,14 +76,6 @@ build: @@ -79,14 +76,6 @@ build:
# MacOSX disallows static executables, so we omit the static flag on this platform
RUN go build -a -installsuffix cgo -ldflags "$([ "$GOOS"z != darwinz ] && echo "-linkmode external -extldflags -static ") -s -w -X github.com/owncast/owncast/config.GitCommit=$EARTHLY_GIT_HASH -X github.com/owncast/owncast/config.VersionNumber=$version -X github.com/owncast/owncast/config.BuildPlatform=$NAME" -tags sqlite_omit_load_extension -o owncast main.go
# Decrease the size of the shipped binary. But only for non-Apple platforms.
# See https://github.com/upx/upx/issues/612
IF [ "$GOOS" != "darwin" ]
RUN upx --best --lzma owncast
# Test the binary
RUN upx -t owncast
END
SAVE ARTIFACT owncast owncast
package:
@ -103,8 +92,6 @@ package: @@ -103,8 +92,6 @@ package:
ARG NAME=linux-arm7
ELSE IF [ "$TARGETPLATFORM" = "darwin/amd64" ]
ARG NAME=macOS-64bit
ELSE IF [ "$TARGETPLATFORM" = "darwin/arm64" ]
ARG NAME=macOS-arm64
ELSE
ARG NAME=custom
END
@ -112,7 +99,7 @@ package: @@ -112,7 +99,7 @@ package:
COPY (+build/owncast --platform $TARGETPLATFORM) /build/dist/owncast
ENV ZIPNAME owncast-$version-$NAME.zip
RUN cd /build/dist && zip -r -q -8 /build/dist/owncast.zip .
SAVE ARTIFACT --keep-ts /build/dist/owncast.zip owncast.zip AS LOCAL dist/$ZIPNAME
SAVE ARTIFACT /build/dist/owncast.zip owncast.zip AS LOCAL dist/$ZIPNAME
docker:
# Multiple image names can be tagged at once. They should all be passed
@ -145,6 +132,12 @@ docker: @@ -145,6 +132,12 @@ docker:
dockerfile:
FROM DOCKERFILE -f Dockerfile .
testing:
ARG images
FOR i IN ${images}
RUN echo "Testing ${i}"
END
unit-tests:
FROM --platform=linux/amd64 bdwyertech/go-crosscompile
COPY . /build

2
LICENSE

@ -1,6 +1,6 @@ @@ -1,6 +1,6 @@
MIT License
Copyright (c) 2020-2023 Gabe Kangas
Copyright (c) 2020 Gabe Kangas
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

18
README.md

@ -48,8 +48,8 @@ Owncast is an open source, self-hosted, decentralized, single user live video st @@ -48,8 +48,8 @@ Owncast is an open source, self-hosted, decentralized, single user live video st
<div>
<img alt="GitHub all releases" src="https://img.shields.io/github/downloads/owncast/owncast/total?style=for-the-badge">
<a href="https://hub.docker.com/r/owncast/owncast">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/owncast/owncast?style=for-the-badge">
<a href="https://hub.docker.com/r/gabekangas/owncast">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/gabekangas/owncast?style=for-the-badge">
</a>
<a href="https://github.com/owncast/owncast/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22">
<img alt="GitHub issues by-label" src="https://img.shields.io/github/issues-raw/owncast/owncast/good%20first%20issue?style=for-the-badge">
@ -69,7 +69,7 @@ The goal is to have a single service that you can run and it works out of the bo @@ -69,7 +69,7 @@ The goal is to have a single service that you can run and it works out of the bo
## Use with your existing broadcasting software
In general, Owncast is compatible with any software that uses `RTMP` to broadcast to a remote server. `RTMP` is what all the major live streaming services use, so if you’re currently using one of those it’s likely that you can point your existing software at your Owncast instance instead.
In general Owncast is compatible with any software that uses `RTMP` to broadcast to a remote server. `RTMP` is what all the major live streaming services use, so if you’re currently using one of those it’s likely that you can point your existing software at your Owncast instance instead.
OBS, Streamlabs, Restream and many others have been used with Owncast. [Read more about compatibility with existing software](https://owncast.online/docs/broadcasting/).
@ -77,8 +77,8 @@ OBS, Streamlabs, Restream and many others have been used with Owncast. [Read mor @@ -77,8 +77,8 @@ OBS, Streamlabs, Restream and many others have been used with Owncast. [Read mor
Owncast consists of two projects.
1. The Owncast backend is written in Go.
1. The frontend is written in React.
1. The Owncast backend written in Go.
1. The frontend written in React.
[Read more about running from source](https://owncast.online/development/).
@ -86,18 +86,16 @@ Owncast consists of two projects. @@ -86,18 +86,16 @@ Owncast consists of two projects.
The `develop` branch is always the most up-to-date state of development and this may not be what you always want. If you want to run the latest released stable version, check out the tag related to that release. For example, if you'd only like the source prior to the v0.1.0 development cycle you can check out the `v0.0.13` tag.
> Note: Currently Owncast does not natively support Windows servers. However, Windows Users can use Windows Subsystem for Linux (WSL2) to install Owncast. For details visit [this document](https://github.com/owncast/owncast/blob/develop/contrib/owncast_for_windows.md).
### Backend
The Owncast backend is a service written in Go.
1. Ensure you have prerequisites installed.
1. Ensure you have pre-requisites installed.
- C compiler, such as [GCC compiler](https://gcc.gnu.org/install/download.html) or a [Musl-compatible compiler](https://musl.libc.org/)
- [ffmpeg](https://ffmpeg.org/download.html)
1. Install the [Go toolchain](https://golang.org/dl/) (1.21 or above).
1. Install the [Go toolchain](https://golang.org/dl/) (1.18 or above).
1. Clone the repo. `git clone https://github.com/owncast/owncast`
1. `go run main.go` will run from the source.
1. `go run main.go` will run from source.
1. Visit `http://yourserver:8080` to access the web interface or `http://yourserver:8080/admin` to access the admin.
1. Point your [broadcasting software](https://owncast.online/docs/broadcasting/) at your new server and start streaming.

16
activitypub/apmodels/actor.go

@ -16,26 +16,26 @@ import ( @@ -16,26 +16,26 @@ import (
// ActivityPubActor represents a single actor in handling ActivityPub activity.
type ActivityPubActor struct {
// RequestObject is the actual follow request object.
RequestObject vocab.ActivityStreamsFollow
// W3IDSecurityV1PublicKey is the public key of the actor.
W3IDSecurityV1PublicKey vocab.W3IDSecurityV1PublicKeyProperty
// ActorIRI is the IRI of the remote actor.
ActorIri *url.URL
// FollowRequestIRI is the unique identifier of the follow request.
FollowRequestIri *url.URL
// Inbox is the inbox URL of the remote follower
Inbox *url.URL
// Image is the avatar image of the Actor.
Image *url.URL
// DisabledAt is the time, if any, this follower was blocked/removed.
DisabledAt *time.Time
// Name is the display name of the follower.
Name string
// Username is the account username of the remote actor.
Username string
// FullUsername is the username@account.tld representation of the user.
FullUsername string
// Image is the avatar image of the Actor.
Image *url.URL
// RequestObject is the actual follow request object.
RequestObject vocab.ActivityStreamsFollow
// W3IDSecurityV1PublicKey is the public key of the actor.
W3IDSecurityV1PublicKey vocab.W3IDSecurityV1PublicKeyProperty
// DisabledAt is the time, if any, this follower was blocked/removed.
DisabledAt *time.Time
}
// DeleteRequest represents a request for delete.

48
activitypub/apmodels/utils.go

@ -4,7 +4,6 @@ import ( @@ -4,7 +4,6 @@ import (
"encoding/json"
"net/url"
"path"
"path/filepath"
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
@ -61,50 +60,3 @@ func Serialize(obj vocab.Type) ([]byte, error) { @@ -61,50 +60,3 @@ func Serialize(obj vocab.Type) ([]byte, error) {
return b, err
}
// MakeLocalIRIForStreamURL will return a full IRI for the local server stream url.
func MakeLocalIRIForStreamURL() *url.URL {
host := data.GetServerURL()
u, err := url.Parse(host)
if err != nil {
log.Errorln("unable to parse local IRI stream url", err)
return nil
}
u.Path = path.Join(u.Path, "/hls/stream.m3u8")
return u
}
// MakeLocalIRIforLogo will return a full IRI for the local server logo.
func MakeLocalIRIforLogo() *url.URL {
host := data.GetServerURL()
u, err := url.Parse(host)
if err != nil {
log.Errorln("unable to parse local IRI stream url", err)
return nil
}
u.Path = path.Join(u.Path, "/logo/external")
return u
}
// GetLogoType will return the rel value for the webfinger response and
// the default static image is of type png.
func GetLogoType() string {
imageFilename := data.GetLogoPath()
if imageFilename == "" {
return "image/png"
}
logoType := "image/jpeg"
if filepath.Ext(imageFilename) == ".svg" {
logoType = "image/svg+xml"
} else if filepath.Ext(imageFilename) == ".gif" {
logoType = "image/gif"
} else if filepath.Ext(imageFilename) == ".png" {
logoType = "image/png"
}
return logoType
}

14
activitypub/apmodels/webfinger.go

@ -26,9 +26,7 @@ type Link struct { @@ -26,9 +26,7 @@ type Link struct {
// MakeWebfingerResponse will create a new Webfinger response.
func MakeWebfingerResponse(account string, inbox string, host string) WebfingerResponse {
accountIRI := MakeLocalIRIForAccount(account)
streamIRI := MakeLocalIRIForStreamURL()
logoIRI := MakeLocalIRIforLogo()
logoType := GetLogoType()
return WebfingerResponse{
Subject: fmt.Sprintf("acct:%s@%s", account, host),
Aliases: []string{
@ -45,16 +43,6 @@ func MakeWebfingerResponse(account string, inbox string, host string) WebfingerR @@ -45,16 +43,6 @@ func MakeWebfingerResponse(account string, inbox string, host string) WebfingerR
Type: "text/html",
Href: accountIRI.String(),
},
{
Rel: "http://webfinger.net/rel/avatar",
Type: logoType,
Href: logoIRI.String(),
},
{
Rel: "alternate",
Type: "application/x-mpegURL",
Href: streamIRI.String(),
},
},
}
}

8
activitypub/controllers/nodeinfo.go

@ -154,13 +154,13 @@ func XNodeInfo2Controller(w http.ResponseWriter, r *http.Request) { @@ -154,13 +154,13 @@ func XNodeInfo2Controller(w http.ResponseWriter, r *http.Request) {
LocalComments int `json:"localComments"`
}
type response struct {
Server Server `json:"server"`
Organization Organization `json:"organization"`
Version string `json:"version"`
Server Server `json:"server"`
Services Services `json:"services"`
Protocols []string `json:"protocols"`
Usage Usage `json:"usage"`
Version string `json:"version"`
OpenRegistrations bool `json:"openRegistrations"`
Usage Usage `json:"usage"`
}
if !data.GetFederationEnabled() {
@ -224,9 +224,9 @@ func InstanceV1Controller(w http.ResponseWriter, r *http.Request) { @@ -224,9 +224,9 @@ func InstanceV1Controller(w http.ResponseWriter, r *http.Request) {
ShortDescription string `json:"short_description"`
Description string `json:"description"`
Version string `json:"version"`
Stats Stats `json:"stats"`
Thumbnail string `json:"thumbnail"`
Languages []string `json:"languages"`
Stats Stats `json:"stats"`
Registrations bool `json:"registrations"`
ApprovalRequired bool `json:"approval_required"`
InvitesEnabled bool `json:"invites_enabled"`

8
activitypub/inbox/like.go

@ -13,14 +13,6 @@ import ( @@ -13,14 +13,6 @@ import (
func handleLikeRequest(c context.Context, activity vocab.ActivityStreamsLike) error {
object := activity.GetActivityStreamsObject()
actorReference := activity.GetActivityStreamsActor()
if object.Len() < 1 {
return errors.New("like activity is missing object")
}
if actorReference.Len() < 1 {
return errors.New("like activity is missing actor")
}
objectIRI := object.At(0).GetIRI().String()
actorIRI := actorReference.At(0).GetIRI().String()

10
activitypub/inbox/workerpool.go

@ -1,14 +1,14 @@ @@ -1,14 +1,14 @@
package inbox
import (
"runtime"
"github.com/owncast/owncast/activitypub/apmodels"
log "github.com/sirupsen/logrus"
)
// workerPoolSize defines the number of concurrent ActivityPub handlers.
var workerPoolSize = runtime.GOMAXPROCS(0)
const (
// InboxWorkerPoolSize defines the number of concurrent ActivityPub handlers.
InboxWorkerPoolSize = 10
)
// Job struct bundling the ActivityPub and the payload in one struct.
type Job struct {
@ -22,7 +22,7 @@ func InitInboxWorkerPool() { @@ -22,7 +22,7 @@ func InitInboxWorkerPool() {
queue = make(chan Job)
// start workers
for i := 1; i <= workerPoolSize; i++ {
for i := 1; i <= InboxWorkerPoolSize; i++ {
go worker(i, queue)
}
}

2
activitypub/outbox/outbox.go

@ -60,7 +60,7 @@ func SendLive() error { @@ -60,7 +60,7 @@ func SendLive() error {
if title := data.GetStreamTitle(); title != "" {
streamTitle = fmt.Sprintf("<p>%s</p>", title)
}
textContent = fmt.Sprintf("<p>%s</p>%s<p>%s</p><p><a href=\"%s\">%s</a></p>", textContent, streamTitle, tagsString, data.GetServerURL(), data.GetServerURL())
textContent = fmt.Sprintf("<p>%s</p>%s<p>%s</p><a href=\"%s\">%s</a>", textContent, streamTitle, tagsString, data.GetServerURL(), data.GetServerURL())
activity, _, note, noteID := createBaseOutboundMessage(textContent)

107
activitypub/persistence/followers_test.go

@ -1,107 +0,0 @@ @@ -1,107 +0,0 @@
package persistence
import (
"os"
"testing"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/utils"
)
func TestMain(m *testing.M) {
setup()
code := m.Run()
os.Exit(code)
}
var followers = []models.Follower{}
func setup() {
data.SetupPersistence(":memory:")
_datastore = data.GetDatastore()
createFederationFollowersTable()
number := 100
for i := 0; i < number; i++ {
u := createFakeFollower()
createFollow(u.ActorIRI, u.Inbox, "https://fake.fediverse.server/some/request", u.Name, u.Username, u.Image, nil, true)
followers = append(followers, u)
}
}
func TestQueryFollowers(t *testing.T) {
f, total, err := GetFederationFollowers(10, 0)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
if len(f) != 10 {
t.Errorf("Expected 10 followers, got %d", len(f))
}
if total != 100 {
t.Errorf("Expected 100 followers, got %d", total)
}
}
func TestQueryFollowersWithOffset(t *testing.T) {
f, total, err := GetFederationFollowers(10, 10)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
if len(f) != 10 {
t.Errorf("Expected 10 followers, got %d", len(f))
}
if total != 100 {
t.Errorf("Expected 100 followers, got %d", total)
}
}
func TestQueryFollowersWithOffsetAndLimit(t *testing.T) {
f, total, err := GetFederationFollowers(10, 90)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
if len(f) != 10 {
t.Errorf("Expected 10 followers, got %d", len(f))
}
if total != 100 {
t.Errorf("Expected 100 followers, got %d", total)
}
}
func TestQueryFollowersWithPagination(t *testing.T) {
f, _, err := GetFederationFollowers(15, 10)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
comparisonFollowers := followers[10:25]
if len(f) != len(comparisonFollowers) {
t.Errorf("Expected %d followers, got %d", len(comparisonFollowers), len(f))
}
for i, follower := range f {
if follower.ActorIRI != comparisonFollowers[i].ActorIRI {
t.Errorf("Expected %s, got %s", comparisonFollowers[i].ActorIRI, follower.ActorIRI)
}
}
}
func createFakeFollower() models.Follower {
user, _ := utils.GenerateRandomString(10)
return models.Follower{
ActorIRI: "https://freedom.eagle/user/" + user,
Inbox: "https://fake.fediverse.server/user/" + user + "/inbox",
Image: "https://fake.fediverse.server/user/" + user + "/avatar.png",
Name: user,
Username: user,
Timestamp: utils.NullTime{},
}
}

4
activitypub/resolvers/resolve.go

@ -3,7 +3,7 @@ package resolvers @@ -3,7 +3,7 @@ package resolvers
import (
"context"
"encoding/json"
"io"
"io/ioutil"
"net/http"
"github.com/go-fed/activity/streams"
@ -63,7 +63,7 @@ func ResolveIRI(c context.Context, iri string, callbacks ...interface{}) error { @@ -63,7 +63,7 @@ func ResolveIRI(c context.Context, iri string, callbacks ...interface{}) error {
defer response.Body.Close()
data, err := io.ReadAll(response.Body)
data, err := ioutil.ReadAll(response.Body)
if err != nil {
return err
}

17
activitypub/webfinger/webfinger.go

@ -2,13 +2,10 @@ package webfinger @@ -2,13 +2,10 @@ package webfinger
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"strings"
"github.com/owncast/owncast/utils"
)
// GetWebfingerLinks will return webfinger data for an account.
@ -21,11 +18,6 @@ func GetWebfingerLinks(account string) ([]map[string]interface{}, error) { @@ -21,11 +18,6 @@ func GetWebfingerLinks(account string) ([]map[string]interface{}, error) {
accountComponents := strings.Split(account, "@")
fediverseServer := accountComponents[1]
// Reject any requests to our internal network or loopback.
if utils.IsHostnameInternal(fediverseServer) {
return nil, errors.New("unable to use provided host as a valid fediverse server")
}
// HTTPS is required.
requestURL, err := url.Parse("https://" + fediverseServer)
if err != nil {
@ -37,14 +29,7 @@ func GetWebfingerLinks(account string) ([]map[string]interface{}, error) { @@ -37,14 +29,7 @@ func GetWebfingerLinks(account string) ([]map[string]interface{}, error) {
query.Add("resource", fmt.Sprintf("acct:%s", account))
requestURL.RawQuery = query.Encode()
// Do not support redirects.
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
response, err := client.Get(requestURL.String())
response, err := http.DefaultClient.Get(requestURL.String())
if err != nil {
return nil, err
}

9
activitypub/workerpool/outbound.go

@ -2,13 +2,14 @@ package workerpool @@ -2,13 +2,14 @@ package workerpool
import (
"net/http"
"runtime"
log "github.com/sirupsen/logrus"
)
// workerPoolSize defines the number of concurrent HTTP ActivityPub requests.
var workerPoolSize = runtime.GOMAXPROCS(0)
const (
// ActivityPubWorkerPoolSize defines the number of concurrent HTTP ActivityPub requests.
ActivityPubWorkerPoolSize = 10
)
// Job struct bundling the ActivityPub and the payload in one struct.
type Job struct {
@ -22,7 +23,7 @@ func InitOutboundWorkerPool() { @@ -22,7 +23,7 @@ func InitOutboundWorkerPool() {
queue = make(chan Job)
// start workers
for i := 1; i <= workerPoolSize; i++ {
for i := 1; i <= ActivityPubWorkerPoolSize; i++ {
go worker(i, queue)
}
}

2
auth/fediverse/fediverse.go

@ -13,11 +13,11 @@ import ( @@ -13,11 +13,11 @@ import (
// OTPRegistration represents a single OTP request.
type OTPRegistration struct {
Timestamp time.Time
UserID string
UserDisplayName string
Code string
Account string
Timestamp time.Time
}
// Key by access token to limit one OTP request for a person

26
auth/indieauth/client.go

@ -12,7 +12,6 @@ import ( @@ -12,7 +12,6 @@ import (
"time"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/utils"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
@ -47,27 +46,10 @@ func setupExpiredRequestPruner() { @@ -47,27 +46,10 @@ func setupExpiredRequestPruner() {
// StartAuthFlow will begin the IndieAuth flow by generating an auth request.
func StartAuthFlow(authHost, userID, accessToken, displayName string) (*url.URL, error) {
// Limit the number of pending requests
if len(pendingAuthRequests) >= maxPendingRequests {
return nil, errors.New("Please try again later. Too many pending requests.")
}
// Reject any requests to our internal network or loopback
if utils.IsHostnameInternal(authHost) {
return nil, errors.New("unable to use provided host")
}
// Santity check the server URL
u, err := url.ParseRequestURI(authHost)
if err != nil {
return nil, errors.New("unable to parse server URL")
}
// Limit to only secured connections
if u.Scheme != "https" {
return nil, errors.New("only servers secured with https are supported")
}
serverURL := data.GetServerURL()
if serverURL == "" {
return nil, errors.New("Owncast server URL must be set when using auth")
@ -98,13 +80,7 @@ func HandleCallbackCode(code, state string) (*Request, *Response, error) { @@ -98,13 +80,7 @@ func HandleCallbackCode(code, state string) (*Request, *Response, error) {
data.Set("redirect_uri", request.Callback.String())
data.Set("code_verifier", request.CodeVerifier)
// Do not support redirects.
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
client := &http.Client{}
r, err := http.NewRequest("POST", request.Endpoint.String(), strings.NewReader(data.Encode())) // URL-encoded payload
if err != nil {
return nil, nil, err

10
auth/indieauth/request.go

@ -7,16 +7,16 @@ import ( @@ -7,16 +7,16 @@ import (
// Request represents a single in-flight IndieAuth request.
type Request struct {
Timestamp time.Time
Endpoint *url.URL
Redirect *url.URL // Outbound redirect URL to continue auth flow
Callback *url.URL // Inbound URL to get auth flow results
Me *url.URL
UserID string
DisplayName string
CurrentAccessToken string
Endpoint *url.URL
Redirect *url.URL // Outbound redirect URL to continue auth flow
Callback *url.URL // Inbound URL to get auth flow results
ClientID string
CodeVerifier string
CodeChallenge string
State string
Me *url.URL
Timestamp time.Time
}

4
auth/indieauth/server.go

@ -12,13 +12,13 @@ import ( @@ -12,13 +12,13 @@ import (
// ServerAuthRequest is n inbound request to authenticate against
// this Owncast instance.
type ServerAuthRequest struct {
Timestamp time.Time
ClientID string
RedirectURI string
CodeChallenge string
State string
Me string
Code string
Timestamp time.Time
}
// ServerProfile represents basic user-provided data about this Owncast instance.
@ -40,7 +40,7 @@ type ServerProfileResponse struct { @@ -40,7 +40,7 @@ type ServerProfileResponse struct {
var pendingServerAuthRequests = map[string]ServerAuthRequest{}
const maxPendingRequests = 100
const maxPendingRequests = 1000
// StartServerAuth will handle the authentication for the admin user of this
// Owncast server. Initiated via a GET of the auth endpoint.

18
build/web/bundleWeb.sh

@ -5,29 +5,17 @@ set -o errexit @@ -5,29 +5,17 @@ set -o errexit
set -o nounset
set -o pipefail
OFFLINE=
while [[ $# -gt 0 ]]; do
case $1 in
--offline)
OFFLINE=1
;;
esac
shift
done
# Change to the root directory of the repository
cd "$(git rev-parse --show-toplevel)"
cd web
if [ ! "$OFFLINE" ]; then
echo "Installing npm modules for the owncast web..."
npm --silent install 2>/dev/null
fi
echo "Installing npm modules for the owncast web..."
npm --silent install 2>/dev/null
echo "Building owncast web..."
rm -rf .next
node_modules/.bin/next build | grep info
(node_modules/.bin/next build && node_modules/.bin/next export) | grep info
echo "Copying web project to dist directory..."

2
config/constants.go

@ -4,7 +4,7 @@ import "path/filepath" @@ -4,7 +4,7 @@ import "path/filepath"
const (
// StaticVersionNumber is the version of Owncast that is used when it's not overwritten via build-time settings.
StaticVersionNumber = "0.1.3" // Shown when you build from develop
StaticVersionNumber = "0.1.0" // Shown when you build from develop
// FfmpegSuggestedVersion is the version of ffmpeg we suggest.
FfmpegSuggestedVersion = "v4.1.5" // Requires the v
// DataDirectory is the directory we save data to.

37
config/defaults.go

@ -8,37 +8,32 @@ import ( @@ -8,37 +8,32 @@ import (
// Defaults will hold default configuration values.
type Defaults struct {
PageBodyContent string
FederationGoLiveMessage string
Name string
Title string
Summary string
ServerWelcomeMessage string
Logo string
YPServer string
Title string
Tags []string
PageBodyContent string
DatabaseFilePath string
WebServerPort int
WebServerIP string
RTMPServerPort int
AdminPassword string
StreamKeys []models.StreamKey
FederationUsername string
WebServerIP string
Name string
AdminPassword string
StreamKeys []models.StreamKey
StreamVariants []models.StreamOutputVariant
Tags []string
RTMPServerPort int
SegmentsInPlaylist int
YPEnabled bool
YPServer string
SegmentLengthSeconds int
WebServerPort int
SegmentsInPlaylist int
StreamVariants []models.StreamOutputVariant
ChatEstablishedUserModeTimeDuration time.Duration
FederationUsername string
FederationGoLiveMessage string
YPEnabled bool
ChatEstablishedUserModeTimeDuration time.Duration
}
// GetDefaults will return default configuration values.

4
config/verifyInstall.go

@ -29,8 +29,8 @@ func VerifyFFMpegPath(path string) error { @@ -29,8 +29,8 @@ func VerifyFFMpegPath(path string) error {
}
mode := stat.Mode()
// source: https://stackoverflow.com/a/60128480
if mode&0o111 == 0 {
//source: https://stackoverflow.com/a/60128480
if mode&0111 == 0 {
return errors.New("ffmpeg path is not executable")
}

82
contrib/owncast_for_windows.md

@ -1,82 +0,0 @@ @@ -1,82 +0,0 @@
# Owncast on Windows
> Note: Owncast currently **does not natively support the Windows Operating System**, however it is possible to run Owncast on Windows using the Windows Subsystem for Linux (WSL2).
This document is a user-contributed document and the Owncast project does not actively maintain Windows support. Hopefully this can be helpful in pointing people in the right direction.
This document list out the steps in detail to install and run Owncast in Windows using Windows Subsystem for Linux, specifically **WSL2**.
Below are steps both for local development, contributing to the project and running it in production.
---
## Required: Installing WSL2 in Windows
There are lots of tutorials available online (videos and docs both) on how to install WSL2.
Here are the official documents from Microsoft -> [Install Linux on Windows with WSL](https://learn.microsoft.com/en-us/windows/wsl/setup/environment)
Some points to remember ->
- Preferable method to install WSL2 is by using the `wsl --install `. If you are facing issues with this method you can look at - [Manual installation steps for older versions of WSL](https://learn.microsoft.com/en-us/windows/wsl/install-manual)
- Make sure you have enabled the Virtual Machine feature. (ignore if used wsl --install method)
- Make sure you have WSL2
- Installed your Linux distribution of choice and make sure you installed the latest available version (Preferably Ubuntu)
### Setting up WSL2 and the distribution of your choice
After basic setup, you can look into setting WSL2 for development. Here is the link for a detailed document by Microsoft - [https://learn.microsoft.com/en-us/windows/wsl/setup/environment](https://learn.microsoft.com/en-us/windows/wsl/setup/environment)
---
## Installing Owncast under WSL2
Once you're running WSL2 in Windows you can install Owncast the same way you would on any Linux distribution by following the [Quickstart](https://owncast.online/quickstart/) guide.
## Contributing to Owncast by performing local development
If you want to use your Windows machine to contribute to Owncast, you'll need to do so under WSL2 and make sure the following prerequisites are installed.
### Make sure all the prerequisites are installed in WSL2
Here is the list for all the prerequisites required ->
- C compiler, such as [GCC compiler](https://gcc.gnu.org/install/download.html) or a [Musl-compatible compiler](https://musl.libc.org/)
- npm (Node Package Manager) is installed as `sudo apt install npm`.
- Node.js is installed (LTS Version) `sudo apt install nodejs`.
- [ffmpeg](https://ffmpeg.org/download.html)
- Install the [Go toolchain](https://golang.org/dl/) (1.21 or above).
### Read more
Once your local development environment is setup, you can read more about how to contribute to Owncast [by reading the development document](https://owncast.online/development/).
## Some possible issues you can face while setting up WSL2
### You have an older version of Nodejs installed in the WSL2
To solve this issue you can look at nvm. Here is one tutorial - [Node-Version-Manager](https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-20-04#option-3-installing-node-using-the-node-version-manager).
### The broadcasting Software failed to connect to the server
This issue arises when you try to use `rtmp://localhost:1935/live` for example in OBS.
To solve this issue you need to find the correct IP address for the WSL2 you are running and use that instead of localhost.
You can use the below commands to find that ->
Note: you can use either of these, whichever works for you.
- In WSL2 Terminal -
`ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}'`
- In Windows Terminal -
`wsl -- ip -o -4 -json addr list eth0`
In this result look for "local": X.X.X.X
After finding the IP address in your broadcasting software make the server point to
`rtmp://<your version of IP address>:1935/live`
Example in OBS-Studio ->
![image](https://user-images.githubusercontent.com/73140257/228762798-a0c56695-c692-4295-b11b-f2e85e867ce7.png)
## More resources
- [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/)
- [Owncast development documentation](https://owncast.online/development/)
- [Owncast quickstart guide](https://owncast.online/quickstart/)
- [Owncast README](https://github.com/owncast/owncast/blob/develop/README.md#building-from-source)

12
controllers/admin/chat.go

@ -31,7 +31,6 @@ func UpdateMessageVisibility(w http.ResponseWriter, r *http.Request) { @@ -31,7 +31,6 @@ func UpdateMessageVisibility(w http.ResponseWriter, r *http.Request) {
}
if r.Method != controllers.POST {
// nolint:goconst
controllers.WriteSimpleResponse(w, false, r.Method+" not supported")
return
}
@ -165,17 +164,12 @@ func UpdateUserEnabled(w http.ResponseWriter, r *http.Request) { @@ -165,17 +164,12 @@ func UpdateUserEnabled(w http.ResponseWriter, r *http.Request) {
disconnectedUser := user.GetUserByID(request.UserID)
_ = chat.SendSystemAction(fmt.Sprintf("**%s** has been removed from chat.", disconnectedUser.DisplayName), true)
localIP4Address := "127.0.0.1"
localIP6Address := "::1"
// Ban this user's IP address.
for _, client := range clients {
ipAddress := client.IPAddress
if ipAddress != localIP4Address && ipAddress != localIP6Address {
reason := fmt.Sprintf("Banning of %s", disconnectedUser.DisplayName)
if err := data.BanIPAddress(ipAddress, reason); err != nil {
log.Errorln("error banning IP address: ", err)
}
reason := fmt.Sprintf("Banning of %s", disconnectedUser.DisplayName)
if err := data.BanIPAddress(ipAddress, reason); err != nil {
log.Errorln("error banning IP address: ", err)
}
}
}

65
controllers/admin/config.go

@ -5,7 +5,6 @@ import ( @@ -5,7 +5,6 @@ import (
"fmt"
"net"
"net/http"
"net/netip"
"os"
"path/filepath"
"reflect"
@ -16,7 +15,6 @@ import ( @@ -16,7 +15,6 @@ import (
"github.com/owncast/owncast/core/chat"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/core/user"
"github.com/owncast/owncast/core/webhooks"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/utils"
log "github.com/sirupsen/logrus"
@ -77,7 +75,6 @@ func SetStreamTitle(w http.ResponseWriter, r *http.Request) { @@ -77,7 +75,6 @@ func SetStreamTitle(w http.ResponseWriter, r *http.Request) {
}
if value != "" {
sendSystemChatAction(fmt.Sprintf("Stream title changed to **%s**", value), true)
go webhooks.SendStreamStatusEvent(models.StreamTitleUpdated)
}
controllers.WriteSimpleResponse(w, true, "changed")
}
@ -407,14 +404,6 @@ func SetServerURL(w http.ResponseWriter, r *http.Request) { @@ -407,14 +404,6 @@ func SetServerURL(w http.ResponseWriter, r *http.Request) {
return
}
// Block Private IP URLs
ipAddr, ipErr := netip.ParseAddr(utils.GetHostnameWithoutPortFromURLString(rawValue))
if ipErr == nil && ipAddr.IsPrivate() {
controllers.WriteSimpleResponse(w, false, "Server URL cannot be private")
return
}
// Trim any trailing slash
serverURL := strings.TrimRight(rawValue, "/")
@ -760,48 +749,6 @@ func SetHideViewerCount(w http.ResponseWriter, r *http.Request) { @@ -760,48 +749,6 @@ func SetHideViewerCount(w http.ResponseWriter, r *http.Request) {
controllers.WriteSimpleResponse(w, true, "hide viewer count setting updated")
}
// SetDisableSearchIndexing will set search indexing support.
func SetDisableSearchIndexing(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
controllers.WriteSimpleResponse(w, false, "unable to update search indexing")
return
}
if err := data.SetDisableSearchIndexing(configValue.Value.(bool)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "search indexing support updated")
}
// SetVideoServingEndpoint will save the video serving endpoint.
func SetVideoServingEndpoint(w http.ResponseWriter, r *http.Request) {
endpoint, success := getValueFromRequest(w, r)
if !success {
controllers.WriteSimpleResponse(w, false, "unable to update custom video serving endpoint")
return
}
value, ok := endpoint.Value.(string)
if !ok {
controllers.WriteSimpleResponse(w, false, "unable to update custom video serving endpoint")
return
}
if err := data.SetVideoServingEndpoint(value); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "custom video serving endpoint updated")
}
func requirePOST(w http.ResponseWriter, r *http.Request) bool {
if r.Method != controllers.POST {
controllers.WriteSimpleResponse(w, false, r.Method+" not supported")
@ -859,18 +806,6 @@ func SetStreamKeys(w http.ResponseWriter, r *http.Request) { @@ -859,18 +806,6 @@ func SetStreamKeys(w http.ResponseWriter, r *http.Request) {
return
}
if len(streamKeys.Value) == 0 {
controllers.WriteSimpleResponse(w, false, "must provide at least one valid stream key")
return
}
for _, streamKey := range streamKeys.Value {
if streamKey.Key == "" {
controllers.WriteSimpleResponse(w, false, "stream key cannot be empty")
return
}
}
if err := data.SetStreamKeys(streamKeys.Value); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return

1
controllers/admin/externalAPIUsers.go

@ -69,6 +69,7 @@ func GetExternalAPIUsers(w http.ResponseWriter, r *http.Request) { @@ -69,6 +69,7 @@ func GetExternalAPIUsers(w http.ResponseWriter, r *http.Request) {
controllers.InternalErrorHandler(w, err)
return
}
controllers.WriteResponse(w, tokens)
}

2
controllers/admin/logs.go

@ -44,9 +44,9 @@ func GetWarnings(w http.ResponseWriter, r *http.Request) { @@ -44,9 +44,9 @@ func GetWarnings(w http.ResponseWriter, r *http.Request) {
}
type logsResponse struct {
Time time.Time `json:"time"`
Message string `json:"message"`
Level string `json:"level"`
Time time.Time `json:"time"`
}
func fromEntry(e *logrus.Entry) logsResponse {

54
controllers/admin/serverConfig.go

@ -52,17 +52,14 @@ func GetServerConfig(w http.ResponseWriter, r *http.Request) { @@ -52,17 +52,14 @@ func GetServerConfig(w http.ResponseWriter, r *http.Request) {
FFmpegPath: ffmpeg,
AdminPassword: data.GetAdminPassword(),
StreamKeys: data.GetStreamKeys(),
StreamKeyOverridden: config.TemporaryStreamKey != "",
WebServerPort: config.WebServerPort,
WebServerIP: config.WebServerIP,
RTMPServerPort: data.GetRTMPPortNumber(),
ChatDisabled: data.GetChatDisabled(),
ChatJoinMessagesEnabled: data.GetChatJoinPartMessagesEnabled(),
ChatJoinMessagesEnabled: data.GetChatJoinMessagesEnabled(),
SocketHostOverride: data.GetWebsocketOverrideHost(),
VideoServingEndpoint: data.GetVideoServingEndpoint(),
ChatEstablishedUserMode: data.GetChatEstbalishedUsersOnlyMode(),
HideViewerCount: data.GetHideViewerCount(),
DisableSearchIndexing: data.GetDisableSearchIndexing(),
VideoSettings: videoSettings{
VideoQualityVariants: videoQualityVariants,
LatencyLevel: data.GetStreamLatencyLevel().Level,
@ -101,29 +98,26 @@ func GetServerConfig(w http.ResponseWriter, r *http.Request) { @@ -101,29 +98,26 @@ func GetServerConfig(w http.ResponseWriter, r *http.Request) {
type serverConfigAdminResponse struct {
InstanceDetails webConfigResponse `json:"instanceDetails"`
Notifications notificationsConfigResponse `json:"notifications"`
YP yp `json:"yp"`
FFmpegPath string `json:"ffmpegPath"`
AdminPassword string `json:"adminPassword"`
SocketHostOverride string `json:"socketHostOverride,omitempty"`
StreamKeys []models.StreamKey `json:"streamKeys"`
WebServerPort int `json:"webServerPort"`
WebServerIP string `json:"webServerIP"`
VideoCodec string `json:"videoCodec"`
VideoServingEndpoint string `json:"videoServingEndpoint"`
RTMPServerPort int `json:"rtmpServerPort"`
S3 models.S3 `json:"s3"`
Federation federationConfigResponse `json:"federation"`
SupportedCodecs []string `json:"supportedCodecs"`
ExternalActions []models.ExternalAction `json:"externalActions"`
ForbiddenUsernames []string `json:"forbiddenUsernames"`
SuggestedUsernames []string `json:"suggestedUsernames"`
StreamKeys []models.StreamKey `json:"streamKeys"`
VideoSettings videoSettings `json:"videoSettings"`
RTMPServerPort int `json:"rtmpServerPort"`
WebServerPort int `json:"webServerPort"`
YP yp `json:"yp"`
ChatDisabled bool `json:"chatDisabled"`
ChatJoinMessagesEnabled bool `json:"chatJoinMessagesEnabled"`
ChatEstablishedUserMode bool `json:"chatEstablishedUserMode"`
DisableSearchIndexing bool `json:"disableSearchIndexing"`
StreamKeyOverridden bool `json:"streamKeyOverridden"`
ExternalActions []models.ExternalAction `json:"externalActions"`
SupportedCodecs []string `json:"supportedCodecs"`
VideoCodec string `json:"videoCodec"`
ForbiddenUsernames []string `json:"forbiddenUsernames"`
Federation federationConfigResponse `json:"federation"`
SuggestedUsernames []string `json:"suggestedUsernames"`
SocketHostOverride string `json:"socketHostOverride,omitempty"`
Notifications notificationsConfigResponse `json:"notifications"`
HideViewerCount bool `json:"hideViewerCount"`
}
@ -133,35 +127,35 @@ type videoSettings struct { @@ -133,35 +127,35 @@ type videoSettings struct {
}
type webConfigResponse struct {
AppearanceVariables map[string]string `json:"appearanceVariables"`
Version string `json:"version"`
Name string `json:"name"`
Summary string `json:"summary"`
WelcomeMessage string `json:"welcomeMessage"`
OfflineMessage string `json:"offlineMessage"`
Logo string `json:"logo"`
Name string `json:"name"`
Tags []string `json:"tags"`
Version string `json:"version"`
NSFW bool `json:"nsfw"`
ExtraPageContent string `json:"extraPageContent"`
StreamTitle string `json:"streamTitle"` // What's going on with the current stream
SocialHandles []models.SocialHandle `json:"socialHandles"`
CustomStyles string `json:"customStyles"`
CustomJavascript string `json:"customJavascript"`
Summary string `json:"summary"`
Tags []string `json:"tags"`
SocialHandles []models.SocialHandle `json:"socialHandles"`
NSFW bool `json:"nsfw"`
AppearanceVariables map[string]string `json:"appearanceVariables"`
}
type yp struct {
Enabled bool `json:"enabled"`
InstanceURL string `json:"instanceUrl"` // The public URL the directory should link to
YPServiceURL string `json:"-"` // The base URL to the YP API to register with (optional)
Enabled bool `json:"enabled"`
}
type federationConfigResponse struct {
Username string `json:"username"`
GoLiveMessage string `json:"goLiveMessage"`
BlockedDomains []string `json:"blockedDomains"`
Enabled bool `json:"enabled"`
IsPrivate bool `json:"isPrivate"`
Username string `json:"username"`
GoLiveMessage string `json:"goLiveMessage"`
ShowEngagement bool `json:"showEngagement"`
BlockedDomains []string `json:"blockedDomains"`
}
type notificationsConfigResponse struct {

8
controllers/admin/status.go

@ -42,11 +42,11 @@ func Status(w http.ResponseWriter, r *http.Request) { @@ -42,11 +42,11 @@ func Status(w http.ResponseWriter, r *http.Request) {
type adminStatusResponse struct {
Broadcaster *models.Broadcaster `json:"broadcaster"`
CurrentBroadcast *models.CurrentBroadcast `json:"currentBroadcast"`
Health *models.StreamHealthOverview `json:"health"`
StreamTitle string `json:"streamTitle"`
VersionNumber string `json:"versionNumber"`
Online bool `json:"online"`
ViewerCount int `json:"viewerCount"`
OverallPeakViewerCount int `json:"overallPeakViewerCount"`
SessionPeakViewerCount int `json:"sessionPeakViewerCount"`
Online bool `json:"online"`
StreamTitle string `json:"streamTitle"`
Health *models.StreamHealthOverview `json:"health"`
VersionNumber string `json:"versionNumber"`
}

3
controllers/admin/update.go

@ -3,6 +3,7 @@ package admin @@ -3,6 +3,7 @@ package admin
import (
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"os/exec"
@ -146,7 +147,7 @@ func getContainerID() string { @@ -146,7 +147,7 @@ func getContainerID() string {
pid := os.Getppid()
cgroupPath := fmt.Sprintf("/proc/%s/cgroup", strconv.Itoa(pid))
containerID := ""
content, err := os.ReadFile(cgroupPath) //nolint:gosec
content, err := ioutil.ReadFile(cgroupPath) //nolint:gosec
if err != nil {
return containerID
}

1
controllers/chat.go

@ -54,7 +54,6 @@ func RegisterAnonymousChatUser(w http.ResponseWriter, r *http.Request) { @@ -54,7 +54,6 @@ func RegisterAnonymousChatUser(w http.ResponseWriter, r *http.Request) {
}
if r.Method != http.MethodPost {
// nolint:goconst
WriteSimpleResponse(w, false, r.Method+" not supported")
return
}

27
controllers/config.go

@ -16,37 +16,36 @@ import ( @@ -16,37 +16,36 @@ import (
)
type webConfigResponse struct {
AppearanceVariables map[string]string `json:"appearanceVariables"`
Name string `json:"name"`
CustomStyles string `json:"customStyles"`
StreamTitle string `json:"streamTitle,omitempty"` // What's going on with the current stream
Summary string `json:"summary"`
OfflineMessage string `json:"offlineMessage"`
Logo string `json:"logo"`
Tags []string `json:"tags"`
Version string `json:"version"`
NSFW bool `json:"nsfw"`
SocketHostOverride string `json:"socketHostOverride,omitempty"`
ExtraPageContent string `json:"extraPageContent"`
Summary string `json:"summary"`
Tags []string `json:"tags"`
StreamTitle string `json:"streamTitle,omitempty"` // What's going on with the current stream
SocialHandles []models.SocialHandle `json:"socialHandles"`
ChatDisabled bool `json:"chatDisabled"`
ExternalActions []models.ExternalAction `json:"externalActions"`
Notifications notificationsConfigResponse `json:"notifications"`
Federation federationConfigResponse `json:"federation"`
CustomStyles string `json:"customStyles"`
AppearanceVariables map[string]string `json:"appearanceVariables"`
MaxSocketPayloadSize int `json:"maxSocketPayloadSize"`
HideViewerCount bool `json:"hideViewerCount"`
ChatDisabled bool `json:"chatDisabled"`
NSFW bool `json:"nsfw"`
Federation federationConfigResponse `json:"federation"`
Notifications notificationsConfigResponse `json:"notifications"`
Authentication authenticationConfigResponse `json:"authentication"`
}
type federationConfigResponse struct {
Enabled bool `json:"enabled"`
Account string `json:"account,omitempty"`
FollowerCount int `json:"followerCount,omitempty"`
Enabled bool `json:"enabled"`
}
type browserNotificationsConfigResponse struct {
PublicKey string `json:"publicKey,omitempty"`
Enabled bool `json:"enabled"`
PublicKey string `json:"publicKey,omitempty"`
}
type notificationsConfigResponse struct {
@ -72,7 +71,6 @@ func GetWebConfig(w http.ResponseWriter, r *http.Request) { @@ -72,7 +71,6 @@ func GetWebConfig(w http.ResponseWriter, r *http.Request) {
func getConfigResponse() webConfigResponse {
pageContent := utils.RenderPageContentMarkdown(data.GetExtraPageBodyContent())
offlineMessage := utils.RenderSimpleMarkdown(data.GetCustomOfflineMessage())
socialHandles := data.GetSocialHandles()
for i, handle := range socialHandles {
platform := models.GetSocialHandle(handle.Platform)
@ -120,7 +118,7 @@ func getConfigResponse() webConfigResponse { @@ -120,7 +118,7 @@ func getConfigResponse() webConfigResponse {
return webConfigResponse{
Name: data.GetServerName(),
Summary: serverSummary,
OfflineMessage: offlineMessage,
OfflineMessage: data.GetCustomOfflineMessage(),
Logo: "/logo",
Tags: data.GetServerMetadataTags(),
Version: config.GetReleaseString(),
@ -137,7 +135,6 @@ func getConfigResponse() webConfigResponse { @@ -137,7 +135,6 @@ func getConfigResponse() webConfigResponse {
Notifications: notificationsResponse,
Authentication: authenticationResponse,
AppearanceVariables: data.GetCustomColorVariableValues(),
HideViewerCount: data.GetHideViewerCount(),
}
}

5
controllers/controllers.go

@ -5,7 +5,6 @@ import ( @@ -5,7 +5,6 @@ import (
"net/http"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/router/middleware"
log "github.com/sirupsen/logrus"
)
@ -56,11 +55,9 @@ func WriteSimpleResponse(w http.ResponseWriter, success bool, message string) { @@ -56,11 +55,9 @@ func WriteSimpleResponse(w http.ResponseWriter, success bool, message string) {
}
}
// WriteResponse will return an object as a JSON encoded uncacheable response.
// WriteResponse will return an object as a JSON encoded response.
func WriteResponse(w http.ResponseWriter, response interface{}) {
w.Header().Set("Content-Type", "application/json")
middleware.DisableCache(w)
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(response); err != nil {

3
controllers/emoji.go

@ -8,13 +8,11 @@ import ( @@ -8,13 +8,11 @@ import (
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/router/middleware"
)
// GetCustomEmojiList returns a list of emoji via the API.
func GetCustomEmojiList(w http.ResponseWriter, r *http.Request) {
emojiList := data.GetEmojiList()
middleware.SetCachingHeaders(w, r)
if err := json.NewEncoder(w).Encode(emojiList); err != nil {
InternalErrorHandler(w, err)
@ -27,6 +25,5 @@ func GetCustomEmojiImage(w http.ResponseWriter, r *http.Request) { @@ -27,6 +25,5 @@ func GetCustomEmojiImage(w http.ResponseWriter, r *http.Request) {
r.URL.Path = path
emojiFS := os.DirFS(config.CustomEmojiPath)
middleware.SetCachingHeaders(w, r)
http.FileServer(http.FS(emojiFS)).ServeHTTP(w, r)
}

98
controllers/index.go

@ -4,18 +4,13 @@ import ( @@ -4,18 +4,13 @@ import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"path/filepath"
"strings"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/router/middleware"
"github.com/owncast/owncast/static"
"github.com/owncast/owncast/utils"
log "github.com/sirupsen/logrus"
)
// IndexHandler handles the default index route.
@ -29,13 +24,6 @@ func IndexHandler(w http.ResponseWriter, r *http.Request) { @@ -29,13 +24,6 @@ func IndexHandler(w http.ResponseWriter, r *http.Request) {
return
}
// For search engine bots and social scrapers return a special
// server-rendered page.
if utils.IsUserAgentABot(r.UserAgent()) && isIndexRequest {
handleScraperMetadataPage(w, r)
return
}
// Set a cache control max-age header
middleware.SetCachingHeaders(w, r)
@ -63,7 +51,6 @@ func renderIndexHtml(w http.ResponseWriter, nonce string) { @@ -63,7 +51,6 @@ func renderIndexHtml(w http.ResponseWriter, nonce string) {
Image string
StatusJSON string
ServerConfigJSON string
EmbedVideo string
Nonce string
}
@ -84,14 +71,13 @@ func renderIndexHtml(w http.ResponseWriter, nonce string) { @@ -84,14 +71,13 @@ func renderIndexHtml(w http.ResponseWriter, nonce string) {
content := serverSideContent{
Name: data.GetServerName(),
Summary: data.GetServerSummary(),
RequestedURL: fmt.Sprintf("%s%s", data.GetServerURL(), "/"),
RequestedURL: data.GetServerURL(),
TagsString: strings.Join(data.GetServerMetadataTags(), ","),
ThumbnailURL: "thumbnail.jpg",
Thumbnail: "thumbnail.jpg",
Image: "logo/external",
ThumbnailURL: "/thumbnail",
Thumbnail: "/thumbnail",
Image: "/logo/external",
StatusJSON: string(sb),
ServerConfigJSON: string(cb),
EmbedVideo: "embed/video",
Nonce: nonce,
}
@ -105,79 +91,3 @@ func renderIndexHtml(w http.ResponseWriter, nonce string) { @@ -105,79 +91,3 @@ func renderIndexHtml(w http.ResponseWriter, nonce string) {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
// MetadataPage represents a server-rendered web page for bots and web scrapers.
type MetadataPage struct {
RequestedURL string
Image string
Thumbnail string
TagsString string
Summary string
Name string
Tags []string
SocialHandles []models.SocialHandle
}
// Return a basic HTML page with server-rendered metadata from the config
// to give to Opengraph clients and web scrapers (bots, web crawlers, etc).
func handleScraperMetadataPage(w http.ResponseWriter, r *http.Request) {
tmpl, err := static.GetBotMetadataTemplate()
if err != nil {
log.Errorln(err)
w.WriteHeader(http.StatusInternalServerError)
return
}
scheme := "http"
if siteURL := data.GetServerURL(); siteURL != "" {
if parsed, err := url.Parse(siteURL); err == nil && parsed.Scheme != "" {
scheme = parsed.Scheme
}
}
fullURL, err := url.Parse(fmt.Sprintf("%s://%s%s", scheme, r.Host, r.URL.Path))
if err != nil {
log.Errorln(err)
}
imageURL, err := url.Parse(fmt.Sprintf("%s://%s%s", scheme, r.Host, "/logo/external"))
if err != nil {
log.Errorln(err)
}
status := core.GetStatus()
// If the thumbnail does not exist or we're offline then just use the logo image
var thumbnailURL string
if status.Online && utils.DoesFileExists(filepath.Join(config.DataDirectory, "tmp", "thumbnail.jpg")) {
thumbnail, err := url.Parse(fmt.Sprintf("%s://%s%s", scheme, r.Host, "/thumbnail.jpg"))
if err != nil {
log.Errorln(err)
thumbnailURL = imageURL.String()
} else {
thumbnailURL = thumbnail.String()
}
} else {
thumbnailURL = imageURL.String()
}
tagsString := strings.Join(data.GetServerMetadataTags(), ",")
metadata := MetadataPage{
Name: data.GetServerName(),
RequestedURL: fullURL.String(),
Image: imageURL.String(),
Summary: data.GetServerSummary(),
Thumbnail: thumbnailURL,
TagsString: tagsString,
Tags: data.GetServerMetadataTags(),
SocialHandles: data.GetSocialHandles(),
}
// Set a cache header
middleware.SetCachingHeaders(w, r)
w.Header().Set("Content-Type", "text/html")
if err := tmpl.Execute(w, metadata); err != nil {
log.Errorln(err)
}
}

6
controllers/moderation/moderation.go

@ -16,11 +16,11 @@ import ( @@ -16,11 +16,11 @@ import (
// GetUserDetails returns the details of a chat user for moderators.
func GetUserDetails(w http.ResponseWriter, r *http.Request) {
type connectedClient struct {
ConnectedAt time.Time `json:"connectedAt"`
UserAgent string `json:"userAgent"`
Geo string `json:"geo,omitempty"`
Id uint `json:"id"`
MessageCount int `json:"messageCount"`
UserAgent string `json:"userAgent"`
ConnectedAt time.Time `json:"connectedAt"`
Geo string `json:"geo,omitempty"`
}
type response struct {

2
controllers/pagination.go

@ -2,6 +2,6 @@ package controllers @@ -2,6 +2,6 @@ package controllers
// PaginatedResponse is a structure for returning a total count with results.
type PaginatedResponse struct {
Results interface{} `json:"results"`
Total int `json:"total"`
Results interface{} `json:"results"`
}

28
controllers/robots.go

@ -1,28 +0,0 @@ @@ -1,28 +0,0 @@
package controllers
import (
"net/http"
"strings"
"github.com/owncast/owncast/core/data"
)
// GetRobotsDotTxt returns the contents of our robots.txt.
func GetRobotsDotTxt(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
contents := []string{
"User-agent: *",
"Disallow: /admin",
"Disallow: /api",
}
if data.GetDisableSearchIndexing() {
contents = append(contents, "Disallow: /")
}
txt := []byte(strings.Join(contents, "\n"))
if _, err := w.Write(txt); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}

4
controllers/status.go

@ -40,12 +40,12 @@ func getStatusResponse() webStatusResponse { @@ -40,12 +40,12 @@ func getStatusResponse() webStatusResponse {
}
type webStatusResponse struct {
Online bool `json:"online"`
ViewerCount int `json:"viewerCount,omitempty"`
ServerTime time.Time `json:"serverTime"`
LastConnectTime *utils.NullTime `json:"lastConnectTime"`
LastDisconnectTime *utils.NullTime `json:"lastDisconnectTime"`
VersionNumber string `json:"versionNumber"`
StreamTitle string `json:"streamTitle"`
ViewerCount int `json:"viewerCount,omitempty"`
Online bool `json:"online"`
}

6
controllers/video.go

@ -8,15 +8,15 @@ import ( @@ -8,15 +8,15 @@ import (
)
type variantsSort struct {
Name string
Index int
VideoBitrate int
Name string
IsVideoPassthrough bool
VideoBitrate int
}
type variantsResponse struct {
Name string `json:"name"`
Index int `json:"index"`
Name string `json:"name"`
}
// GetVideoStreamOutputVariants will return the video variants available.

28
core/chat/chatclient.go

@ -19,27 +19,27 @@ import ( @@ -19,27 +19,27 @@ import (
// Client represents a single chat client.
type Client struct {
ConnectedAt time.Time `json:"connectedAt"`
timeoutTimer *time.Timer
rateLimiter *rate.Limiter
conn *websocket.Conn
User *user.User `json:"user"`
server *Server
Geo *geoip.GeoDetails `json:"geo"`
mu sync.RWMutex
Id uint `json:"-"`
accessToken string
conn *websocket.Conn
User *user.User `json:"user"`
server *Server
IPAddress string `json:"-"`
// Buffered channel of outbound messages.
send chan []byte
accessToken string
IPAddress string `json:"-"`
UserAgent string `json:"userAgent"`
MessageCount int `json:"messageCount"`
Id uint `json:"-"`
mu sync.RWMutex
rateLimiter *rate.Limiter
timeoutTimer *time.Timer
inTimeout bool
Geo *geoip.GeoDetails `json:"geo"`
MessageCount int `json:"messageCount"`
UserAgent string `json:"userAgent"`
ConnectedAt time.Time `json:"connectedAt"`
}
type chatClientEvent struct {
client *Client
data []byte
client *Client
}
const (

1
core/chat/concurrentConnections_freebsd.go

@ -1,4 +1,3 @@ @@ -1,4 +1,3 @@
//go:build freebsd
// +build freebsd
package chat

7
core/chat/events.go

@ -63,12 +63,6 @@ func (s *Server) userNameChanged(eventData chatClientEvent) { @@ -63,12 +63,6 @@ func (s *Server) userNameChanged(eventData chatClientEvent) {
savedUser := user.GetUserByToken(eventData.client.accessToken)
oldName := savedUser.DisplayName
// Check that the new name is different from old.
if proposedUsername == oldName {
eventData.client.sendConnectedClientInfo()
return
}
// Save the new name
if err := user.ChangeUsername(eventData.client.User.ID, proposedUsername); err != nil {
log.Errorln("error changing username", err)
@ -167,6 +161,7 @@ func (s *Server) userMessageSent(eventData chatClientEvent) { @@ -167,6 +161,7 @@ func (s *Server) userMessageSent(eventData chatClientEvent) {
SaveUserMessage(event)
eventData.client.MessageCount++
_lastSeenCache[event.User.ID] = time.Now()
}
func logSanitize(userValue string) string {

2
core/chat/events/connectedClientInfo.go

@ -4,6 +4,6 @@ import "github.com/owncast/owncast/core/user" @@ -4,6 +4,6 @@ import "github.com/owncast/owncast/core/user"
// ConnectedClientInfo represents the information about a connected client.
type ConnectedClientInfo struct {
User *user.User `json:"user"`
Event
User *user.User `json:"user"`
}

135
core/chat/events/events.go

@ -4,22 +4,15 @@ import ( @@ -4,22 +4,15 @@ import (
"bytes"
"regexp"
"strings"
"sync"
"text/template"
"time"
"github.com/microcosm-cc/bluemonday"
"github.com/teris-io/shortid"
"github.com/yuin/goldmark"
emoji "github.com/yuin/goldmark-emoji"
emojiAst "github.com/yuin/goldmark-emoji/ast"
emojiDef "github.com/yuin/goldmark-emoji/definition"
"github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/renderer/html"
"github.com/yuin/goldmark/util"
"mvdan.cc/xurls"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/core/user"
log "github.com/sirupsen/logrus"
)
@ -35,16 +28,16 @@ type OutboundEvent interface { @@ -35,16 +28,16 @@ type OutboundEvent interface {
// Event is any kind of event. A type is required to be specified.
type Event struct {
Timestamp time.Time `json:"timestamp"`
Type EventType `json:"type,omitempty"`
ID string `json:"id"`
Timestamp time.Time `json:"timestamp"`
}
// UserEvent is an event with an associated user.
type UserEvent struct {
User *user.User `json:"user"`
HiddenAt *time.Time `json:"hiddenAt,omitempty"`
ClientID uint `json:"clientId,omitempty"`
HiddenAt *time.Time `json:"hiddenAt,omitempty"`
}
// MessageEvent is an event that has a message body.
@ -73,105 +66,6 @@ func (e *UserMessageEvent) SetDefaults() { @@ -73,105 +66,6 @@ func (e *UserMessageEvent) SetDefaults() {
e.RenderAndSanitizeMessageBody()
}
// implements the emojiDef.Emojis interface but uses case-insensitive search.
// the .children field isn't currently used, but could be used in a future
// implementation of say, emoji packs where a child represents a pack.
type emojis struct {
list []emojiDef.Emoji
names map[string]*emojiDef.Emoji
children []emojiDef.Emojis
}
// return a new Emojis set.
func newEmojis(emotes ...emojiDef.Emoji) emojiDef.Emojis {
self := &emojis{
list: emotes,
names: map[string]*emojiDef.Emoji{},
children: []emojiDef.Emojis{},
}
for i := range self.list {
emoji := &self.list[i]
for _, s := range emoji.ShortNames {
self.names[s] = emoji
}
}
return self
}
func (self *emojis) Get(shortName string) (*emojiDef.Emoji, bool) {
v, ok := self.names[strings.ToLower(shortName)]
if ok {
return v, ok
}
for _, child := range self.children {
v, ok := child.Get(shortName)
if ok {
return v, ok
}
}
return nil, false
}
func (self *emojis) Add(emotes emojiDef.Emojis) {
self.children = append(self.children, emotes)
}
func (self *emojis) Clone() emojiDef.Emojis {
clone := &emojis{
list: self.list,
names: self.names,
children: make([]emojiDef.Emojis, len(self.children)),
}
copy(clone.children, self.children)
return clone
}
var (
emojiMu sync.Mutex
emojiDefs = newEmojis()
emojiHTML = make(map[string]string)
emojiModTime time.Time
emojiHTMLFormat = `<img src="{{ .URL }}" class="emoji" alt=":{{ .Name }}:" title=":{{ .Name }}:">`
emojiHTMLTemplate = template.Must(template.New("emojiHTML").Parse(emojiHTMLFormat))
)
func loadEmoji() {
modTime, err := data.UpdateEmojiList(false)
if err != nil {
return
}
if modTime.After(emojiModTime) {
emojiMu.Lock()
defer emojiMu.Unlock()
emojiHTML = make(map[string]string)
emojiList := data.GetEmojiList()
emojiArr := make([]emojiDef.Emoji, 0)
for i := 0; i < len(emojiList); i++ {
var buf bytes.Buffer
err := emojiHTMLTemplate.Execute(&buf, emojiList[i])
if err != nil {
return
}
emojiHTML[strings.ToLower(emojiList[i].Name)] = buf.String()
emoji := emojiDef.NewEmoji(emojiList[i].Name, nil, strings.ToLower(emojiList[i].Name))
emojiArr = append(emojiArr, emoji)
}
emojiDefs = newEmojis(emojiArr...)
}
}
// RenderAndSanitizeMessageBody will turn markdown into HTML, sanitize raw user-supplied HTML and standardize
// the message into something safe and renderable for clients.
func (m *MessageEvent) RenderAndSanitizeMessageBody() {
@ -204,11 +98,6 @@ func RenderAndSanitize(raw string) string { @@ -204,11 +98,6 @@ func RenderAndSanitize(raw string) string {
// RenderMarkdown will return HTML rendered from the string body of a chat message.
func RenderMarkdown(raw string) string {
loadEmoji()
emojiMu.Lock()
defer emojiMu.Unlock()
markdown := goldmark.New(
goldmark.WithRendererOptions(
html.WithUnsafe(),
@ -223,16 +112,6 @@ func RenderMarkdown(raw string) string { @@ -223,16 +112,6 @@ func RenderMarkdown(raw string) string {
xurls.Strict,
),
),
emoji.New(
emoji.WithEmojis(
emojiDefs,
),
emoji.WithRenderingMethod(emoji.Func),
emoji.WithRendererFunc(func(w util.BufWriter, source []byte, n *emojiAst.Emoji, config *emoji.RendererConfig) {
baseName := n.Value.ShortNames[0]
_, _ = w.WriteString(emojiHTML[baseName])
}),
),
),
)
@ -246,9 +125,9 @@ func RenderMarkdown(raw string) string { @@ -246,9 +125,9 @@ func RenderMarkdown(raw string) string {
}
var (
_sanitizeReSrcMatch = regexp.MustCompile(`(?i)^/img/emoji/[^\.%]*.[A-Z]*$`)
_sanitizeReClassMatch = regexp.MustCompile(`(?i)^(emoji)[A-Z_]*?$`)
_sanitizeNonEmptyMatch = regexp.MustCompile(`^.+$`)
_sanitizeReSrcMatch = regexp.MustCompile(`(?i)^/img/emoji/[^\.%]*.[A-Z]*$`)
_sanitizeReAltTitleMatch = regexp.MustCompile(`:\S+:`)
_sanitizeReClassMatch = regexp.MustCompile(`(?i)^(emoji)[A-Z_]*?$`)
)
func sanitize(raw string) string {
@ -270,11 +149,11 @@ func sanitize(raw string) string { @@ -270,11 +149,11 @@ func sanitize(raw string) string {
// Allow breaks
p.AllowElements("br")
p.AllowElements("p")
p.AllowElementsContent("p")
// Allow img tags from the the local emoji directory only
p.AllowAttrs("src").Matching(_sanitizeReSrcMatch).OnElements("img")
p.AllowAttrs("alt", "title").Matching(_sanitizeNonEmptyMatch).OnElements("img")
p.AllowAttrs("alt", "title").Matching(_sanitizeReAltTitleMatch).OnElements("img")
p.AllowAttrs("class").Matching(_sanitizeReClassMatch).OnElements("img")
// Allow bold

6
core/chat/events/eventtype.go

@ -8,8 +8,6 @@ const ( @@ -8,8 +8,6 @@ const (
MessageSent EventType = "CHAT"
// UserJoined is the event sent when a chat user join action takes place.
UserJoined EventType = "USER_JOINED"
// UserParted is the event sent when a chat user part action takes place.
UserParted EventType = "USER_PARTED"
// UserNameChanged is the event sent when a chat username change takes place.
UserNameChanged EventType = "NAME_CHANGE"
// UserColorChanged is the event sent when a chat user color change takes place.
@ -24,8 +22,6 @@ const ( @@ -24,8 +22,6 @@ const (
StreamStarted EventType = "STREAM_STARTED"
// StreamStopped represents a stream stopped event.
StreamStopped EventType = "STREAM_STOPPED"
// StreamTitleUpdated is the event sent when a stream's title changes.
StreamTitleUpdated EventType = "STREAM_TITLE_UPDATED"
// SystemMessageSent is the event sent when a system message is sent.
SystemMessageSent EventType = "SYSTEM"
// ChatDisabled is when a user is explicitly disabled and blocked from using chat.
@ -35,7 +31,7 @@ const ( @@ -35,7 +31,7 @@ const (
// ChatActionSent is a generic chat action that can be used for anything that doesn't need specific handling or formatting.
ChatActionSent EventType = "CHAT_ACTION"
// ErrorNeedsRegistration is an error returned when the client needs to perform registration.
ErrorNeedsRegistration EventType = "ERROR_NEEDS_REGISTRATION" // nolint:gosec
ErrorNeedsRegistration EventType = "ERROR_NEEDS_REGISTRATION"
// ErrorMaxConnectionsExceeded is an error returned when the server determined it should not handle more connections.
ErrorMaxConnectionsExceeded EventType = "ERROR_MAX_CONNECTIONS_EXCEEDED"
// ErrorUserDisabled is an error returned when the connecting user has been previously banned/disabled.

17
core/chat/events/userPartEvent.go

@ -1,17 +0,0 @@ @@ -1,17 +0,0 @@
package events
// UserPartEvent is the event fired when a user leaves chat.
type UserPartEvent struct {
Event
UserEvent
}
// GetBroadcastPayload will return the object to send to all chat users.
func (e *UserPartEvent) GetBroadcastPayload() EventPayload {
return EventPayload{
"type": UserParted,
"id": e.ID,
"timestamp": e.Timestamp,
"user": e.User,
}
}

25
core/chat/messageRendering_test.go

@ -10,17 +10,20 @@ import ( @@ -10,17 +10,20 @@ import (
// and fully rendered HTML out of it.
func TestRenderAndSanitize(t *testing.T) {
messageContent := `
Test one two three! I go to http://yahoo.com and search for _sports_ and **answers**.
Here is an iframe<iframe src="http://yahoo.com"></iframe>
## blah blah blah
[test link](http://owncast.online)
<img class="emoji" src="/img/emoji/bananadance.gif">`
Test one two three! I go to http://yahoo.com and search for _sports_ and **answers**.
Here is an iframe <iframe src="http://yahoo.com"></iframe>
## blah blah blah
[test link](http://owncast.online)
<img class="emoji" alt="bananadance.gif" width="600px" src="/img/emoji/bananadance.gif">
<script src="http://hackers.org/hack.js"></script>
`
expected := `<p>Test one two three! I go to <a href="http://yahoo.com" rel="nofollow noreferrer noopener" target="_blank">http://yahoo.com</a> and search for <em>sports</em> and <strong>answers</strong>.
Here is an iframe</p>
expected := `Test one two three! I go to <a href="http://yahoo.com" rel="nofollow noreferrer noopener" target="_blank">http://yahoo.com</a> and search for <em>sports</em> and <strong>answers</strong>.
Here is an iframe
blah blah blah
<p><a href="http://owncast.online" rel="nofollow noreferrer noopener" target="_blank">test link</a>
<img class="emoji" src="/img/emoji/bananadance.gif"></p>`
<a href="http://owncast.online" rel="nofollow noreferrer noopener" target="_blank">test link</a>
<img class="emoji" src="/img/emoji/bananadance.gif">`
result := events.RenderAndSanitize(messageContent)
if result != expected {
@ -31,7 +34,7 @@ blah blah blah @@ -31,7 +34,7 @@ blah blah blah
// Test to make sure we block remote images in chat messages.
func TestBlockRemoteImages(t *testing.T) {
messageContent := `<img src="https://via.placeholder.com/img/emoji/350x150"> test ![](https://via.placeholder.com/img/emoji/350x150)`
expected := `<p> test </p>`
expected := `test`
result := events.RenderAndSanitize(messageContent)
if result != expected {
@ -42,7 +45,7 @@ func TestBlockRemoteImages(t *testing.T) { @@ -42,7 +45,7 @@ func TestBlockRemoteImages(t *testing.T) {
// Test to make sure emoji images are allowed in chat messages.
func TestAllowEmojiImages(t *testing.T) {
messageContent := `<img alt=":beerparrot:" title=":beerparrot:" src="/img/emoji/beerparrot.gif"> test ![](/img/emoji/beerparrot.gif)`
expected := `<p><img alt=":beerparrot:" title=":beerparrot:" src="/img/emoji/beerparrot.gif"> test <img src="/img/emoji/beerparrot.gif"></p>`
expected := `<img alt=":beerparrot:" title=":beerparrot:" src="/img/emoji/beerparrot.gif"> test <img src="/img/emoji/beerparrot.gif">`
result := events.RenderAndSanitize(messageContent)
if result != expected {

35
core/chat/persistence.go

@ -186,27 +186,26 @@ func makeFederatedActionChatEventFromRowData(row rowData) events.FediverseEngage @@ -186,27 +186,26 @@ func makeFederatedActionChatEventFromRowData(row rowData) events.FediverseEngage
}
type rowData struct {
timestamp time.Time
image *string
previousUsernames *string
userDisplayName *string
userDisplayColor *int
userID *string
title *string
subtitle *string
link *string
userType *string
userScopes *string
hiddenAt *time.Time
id string
userID *string
body string
eventType models.EventType
hiddenAt *time.Time
timestamp time.Time
title *string
subtitle *string
image *string
link *string
userDisplayName *string
userDisplayColor *int
userCreatedAt *time.Time
userDisabledAt *time.Time
userAuthenticatedAt *time.Time
previousUsernames *string
userNameChangedAt *time.Time
body string
eventType models.EventType
id string
userAuthenticatedAt *time.Time
userScopes *string
userType *string
}
func getChat(rows *sql.Rows) ([]interface{}, error) {

79
core/chat/server.go

@ -22,9 +22,15 @@ import ( @@ -22,9 +22,15 @@ import (
var _server *Server
// a map of user IDs and when they last were active.
var _lastSeenCache = map[string]time.Time{}
// Server represents an instance of the chat server.
type Server struct {
clients map[uint]*Client
mu sync.RWMutex
seq uint
clients map[uint]*Client
maxSocketConnectionLimit int64
// send outbound message payload to all clients
outbound chan []byte
@ -36,13 +42,6 @@ type Server struct { @@ -36,13 +42,6 @@ type Server struct {
unregister chan uint // the ChatClient id
geoipClient *geoip.Client
// a map of user IDs and timers that fire for chat part messages.
userPartedTimers map[string]*time.Ticker
seq uint
maxSocketConnectionLimit int64
mu sync.RWMutex
}
// NewChat will return a new instance of the chat server.
@ -57,7 +56,6 @@ func NewChat() *Server { @@ -57,7 +56,6 @@ func NewChat() *Server {
unregister: make(chan uint),
maxSocketConnectionLimit: maximumConcurrentConnectionLimit,
geoipClient: geoip.NewClient(),
userPartedTimers: map[string]*time.Ticker{},
}
return server
@ -68,8 +66,7 @@ func (s *Server) Run() { @@ -68,8 +66,7 @@ func (s *Server) Run() {
for {
select {
case clientID := <-s.unregister:
if client, ok := s.clients[clientID]; ok {
s.handleClientDisconnected(client)
if _, ok := s.clients[clientID]; ok {
s.mu.Lock()
delete(s.clients, clientID)
s.mu.Unlock()
@ -94,22 +91,18 @@ func (s *Server) Addclient(conn *websocket.Conn, user *user.User, accessToken st @@ -94,22 +91,18 @@ func (s *Server) Addclient(conn *websocket.Conn, user *user.User, accessToken st
ConnectedAt: time.Now(),
}
shouldSendJoinedMessages := data.GetChatJoinPartMessagesEnabled()
// Do not send user re-joined broadcast message if they've been active within 5 minutes.
shouldSendJoinedMessages := data.GetChatJoinMessagesEnabled()
if previouslyLastSeen, ok := _lastSeenCache[user.ID]; ok && time.Since(previouslyLastSeen) < time.Minute*5 {
shouldSendJoinedMessages = false
}
s.mu.Lock()
{
// If there is a pending disconnect timer then clear it.
// Do not send user joined message if enough time hasn't passed where the
// user chat part message hasn't been sent yet.
if ticker, ok := s.userPartedTimers[user.ID]; ok {
ticker.Stop()
delete(s.userPartedTimers, user.ID)
shouldSendJoinedMessages = false
}
client.Id = s.seq
s.clients[client.Id] = client
s.seq++
_lastSeenCache[user.ID] = time.Now()
}
s.mu.Unlock()
@ -149,43 +142,16 @@ func (s *Server) sendUserJoinedMessage(c *Client) { @@ -149,43 +142,16 @@ func (s *Server) sendUserJoinedMessage(c *Client) {
webhooks.SendChatEventUserJoined(userJoinedEvent)
}
func (s *Server) handleClientDisconnected(c *Client) {
// ClientClosed is fired when a client disconnects or connection is dropped.
func (s *Server) ClientClosed(c *Client) {
s.mu.Lock()
defer s.mu.Unlock()
c.close()
if _, ok := s.clients[c.Id]; ok {
log.Debugln("Deleting", c.Id)
delete(s.clients, c.Id)
}
additionalClientCheck, _ := GetClientsForUser(c.User.ID)
if len(additionalClientCheck) > 0 {
// This user is still connected to chat with another client.
return
}
s.userPartedTimers[c.User.ID] = time.NewTicker(10 * time.Second)
go func() {
<-s.userPartedTimers[c.User.ID].C
s.sendUserPartedMessage(c)
}()
}
func (s *Server) sendUserPartedMessage(c *Client) {
s.userPartedTimers[c.User.ID].Stop()
delete(s.userPartedTimers, c.User.ID)
userPartEvent := events.UserPartEvent{}
userPartEvent.SetDefaults()
userPartEvent.User = c.User
userPartEvent.ClientID = c.Id
// If part messages are disabled.
if data.GetChatJoinPartMessagesEnabled() {
if err := s.Broadcast(userPartEvent.GetBroadcastPayload()); err != nil {
log.Errorln("error sending chat part message", err)
}
}
// Send chat user joined webhook
webhooks.SendChatEventUserParted(userPartEvent)
}
// HandleClientConnection is fired when a single client connects to the websocket.
@ -199,8 +165,13 @@ func (s *Server) HandleClientConnection(w http.ResponseWriter, r *http.Request) @@ -199,8 +165,13 @@ func (s *Server) HandleClientConnection(w http.ResponseWriter, r *http.Request)
// Check if this client's IP address is banned. If so send a rejection.
if blocked, err := data.IsIPAddressBanned(ipAddress); blocked {
log.Debugln("Client ip address has been blocked. Rejecting.")
event := events.UserDisabledEvent{}
event.SetDefaults()
w.WriteHeader(http.StatusForbidden)
// Send this disabled event specifically to this single connected client
// to let them know they've been banned.
// _server.Send(event.GetBroadcastPayload(), client)
return
} else if err != nil {
log.Errorln("error determining if IP address is blocked: ", err)

8
core/core.go

@ -76,11 +76,9 @@ func Start() error { @@ -76,11 +76,9 @@ func Start() error {
go rtmp.Start(setStreamAsConnected, setBroadcaster)
rtmpPort := data.GetRTMPPortNumber()
if rtmpPort != 1935 {
log.Infof("RTMP is accepting inbound streams on port %d.", rtmpPort)
}
log.Infof("RTMP is accepting inbound streams on port %d.", rtmpPort)
webhooks.SetupWebhooks(GetStatus)
webhooks.InitWorkerPool()
notifications.Setup(data.GetStore())
@ -110,7 +108,7 @@ func transitionToOfflineVideoStreamContent() { @@ -110,7 +108,7 @@ func transitionToOfflineVideoStreamContent() {
}
_transcoder.SetInput(offlineFilePath)
go _transcoder.Start(false)
go _transcoder.Start()
// Copy the logo to be the thumbnail
logo := data.GetLogoPath()

138
core/data/config.go

@ -16,62 +16,59 @@ import ( @@ -16,62 +16,59 @@ import (
)
const (
extraContentKey = "extra_page_content"
streamTitleKey = "stream_title"
adminPasswordKey = "admin_password_key"
logoPathKey = "logo_path"
logoUniquenessKey = "logo_uniqueness"
serverSummaryKey = "server_summary"
serverWelcomeMessageKey = "server_welcome_message"
serverNameKey = "server_name"
serverURLKey = "server_url"
httpPortNumberKey = "http_port_number"
httpListenAddressKey = "http_listen_address"
websocketHostOverrideKey = "websocket_host_override"
rtmpPortNumberKey = "rtmp_port_number"
serverMetadataTagsKey = "server_metadata_tags"
directoryEnabledKey = "directory_enabled"
directoryRegistrationKeyKey = "directory_registration_key"
socialHandlesKey = "social_handles"
peakViewersSessionKey = "peak_viewers_session"
peakViewersOverallKey = "peak_viewers_overall"
lastDisconnectTimeKey = "last_disconnect_time"
ffmpegPathKey = "ffmpeg_path"
nsfwKey = "nsfw"
s3StorageConfigKey = "s3_storage_config"
videoLatencyLevel = "video_latency_level"
videoStreamOutputVariantsKey = "video_stream_output_variants"
chatDisabledKey = "chat_disabled"
externalActionsKey = "external_actions"
customStylesKey = "custom_styles"
customJavascriptKey = "custom_javascript"
videoCodecKey = "video_codec"
blockedUsernamesKey = "blocked_usernames"
publicKeyKey = "public_key"
privateKeyKey = "private_key"
serverInitDateKey = "server_init_date"
federationEnabledKey = "federation_enabled"
federationUsernameKey = "federation_username"
federationPrivateKey = "federation_private"
federationGoLiveMessageKey = "federation_go_live_message"
federationShowEngagementKey = "federation_show_engagement"
federationBlockedDomainsKey = "federation_blocked_domains"
suggestedUsernamesKey = "suggested_usernames"
chatJoinMessagesEnabledKey = "chat_join_messages_enabled"
chatEstablishedUsersOnlyModeKey = "chat_established_users_only_mode"
notificationsEnabledKey = "notifications_enabled"
discordConfigurationKey = "discord_configuration"
browserPushConfigurationKey = "browser_push_configuration"
browserPushPublicKeyKey = "browser_push_public_key"
// nolint:gosec
extraContentKey = "extra_page_content"
streamTitleKey = "stream_title"
adminPasswordKey = "admin_password_key"
logoPathKey = "logo_path"
logoUniquenessKey = "logo_uniqueness"
serverSummaryKey = "server_summary"
serverWelcomeMessageKey = "server_welcome_message"
serverNameKey = "server_name"
serverURLKey = "server_url"
httpPortNumberKey = "http_port_number"
httpListenAddressKey = "http_listen_address"
websocketHostOverrideKey = "websocket_host_override"
rtmpPortNumberKey = "rtmp_port_number"
serverMetadataTagsKey = "server_metadata_tags"
directoryEnabledKey = "directory_enabled"
directoryRegistrationKeyKey = "directory_registration_key"
socialHandlesKey = "social_handles"
peakViewersSessionKey = "peak_viewers_session"
peakViewersOverallKey = "peak_viewers_overall"
lastDisconnectTimeKey = "last_disconnect_time"
ffmpegPathKey = "ffmpeg_path"
nsfwKey = "nsfw"
s3StorageConfigKey = "s3_storage_config"
videoLatencyLevel = "video_latency_level"
videoStreamOutputVariantsKey = "video_stream_output_variants"
chatDisabledKey = "chat_disabled"
externalActionsKey = "external_actions"
customStylesKey = "custom_styles"
customJavascriptKey = "custom_javascript"
videoCodecKey = "video_codec"
blockedUsernamesKey = "blocked_usernames"
publicKeyKey = "public_key"
privateKeyKey = "private_key"
serverInitDateKey = "server_init_date"
federationEnabledKey = "federation_enabled"
federationUsernameKey = "federation_username"
federationPrivateKey = "federation_private"
federationGoLiveMessageKey = "federation_go_live_message"
federationShowEngagementKey = "federation_show_engagement"
federationBlockedDomainsKey = "federation_blocked_domains"
suggestedUsernamesKey = "suggested_usernames"
chatJoinMessagesEnabledKey = "chat_join_messages_enabled"
chatEstablishedUsersOnlyModeKey = "chat_established_users_only_mode"
notificationsEnabledKey = "notifications_enabled"
discordConfigurationKey = "discord_configuration"
browserPushConfigurationKey = "browser_push_configuration"
browserPushPublicKeyKey = "browser_push_public_key"
browserPushPrivateKeyKey = "browser_push_private_key"
hasConfiguredInitialNotificationsKey = "has_configured_initial_notifications"
hideViewerCountKey = "hide_viewer_count"
customOfflineMessageKey = "custom_offline_message"
customColorVariableValuesKey = "custom_color_variable_values"
streamKeysKey = "stream_keys"
disableSearchIndexingKey = "disable_search_indexing"
videoServingEndpointKey = "video_serving_endpoint"
)
// GetExtraPageBodyContent will return the user-supplied body content.
@ -620,19 +617,19 @@ func VerifySettings() error { @@ -620,19 +617,19 @@ func VerifySettings() error {
}
// FindHighestVideoQualityIndex will return the highest quality from a slice of variants.
func FindHighestVideoQualityIndex(qualities []models.StreamOutputVariant) (int, bool) {
func FindHighestVideoQualityIndex(qualities []models.StreamOutputVariant) int {
type IndexedQuality struct {
quality models.StreamOutputVariant
index int
quality models.StreamOutputVariant
}
if len(qualities) < 2 {
return 0, qualities[0].IsVideoPassthrough
return 0
}
indexedQualities := make([]IndexedQuality, 0)
for index, quality := range qualities {
indexedQuality := IndexedQuality{quality, index}
indexedQuality := IndexedQuality{index, quality}
indexedQualities = append(indexedQualities, indexedQuality)
}
@ -648,9 +645,7 @@ func FindHighestVideoQualityIndex(qualities []models.StreamOutputVariant) (int, @@ -648,9 +645,7 @@ func FindHighestVideoQualityIndex(qualities []models.StreamOutputVariant) (int,
return indexedQualities[a].quality.VideoBitrate > indexedQualities[b].quality.VideoBitrate
})
// nolint:gosec
selectedQuality := indexedQualities[0]
return selectedQuality.index, selectedQuality.quality.IsVideoPassthrough
return indexedQualities[0].index
}
// GetForbiddenUsernameList will return the blocked usernames as a comma separated string.
@ -817,8 +812,8 @@ func SetChatJoinMessagesEnabled(enabled bool) error { @@ -817,8 +812,8 @@ func SetChatJoinMessagesEnabled(enabled bool) error {
return _datastore.SetBool(chatJoinMessagesEnabledKey, enabled)
}
// GetChatJoinPartMessagesEnabled will return if chat join messages are enabled.
func GetChatJoinPartMessagesEnabled() bool {
// GetChatJoinMessagesEnabled will return if chat join messages are enabled.
func GetChatJoinMessagesEnabled() bool {
enabled, err := _datastore.GetBool(chatJoinMessagesEnabledKey)
if err != nil {
return true
@ -964,28 +959,3 @@ func SetStreamKeys(actions []models.StreamKey) error { @@ -964,28 +959,3 @@ func SetStreamKeys(actions []models.StreamKey) error {
configEntry := ConfigEntry{Key: streamKeysKey, Value: actions}
return _datastore.Save(configEntry)
}
// SetDisableSearchIndexing will set if the web server should be indexable.
func SetDisableSearchIndexing(disableSearchIndexing bool) error {
return _datastore.SetBool(disableSearchIndexingKey, disableSearchIndexing)
}
// GetDisableSearchIndexing will return if the web server should be indexable.
func GetDisableSearchIndexing() bool {
disableSearchIndexing, err := _datastore.GetBool(disableSearchIndexingKey)
if err != nil {
return false
}
return disableSearchIndexing
}
// GetVideoServingEndpoint returns the custom video endpont.
func GetVideoServingEndpoint() string {
message, _ := _datastore.GetString(videoServingEndpointKey)
return message
}
// SetVideoServingEndpoint sets the custom video endpoint.
func SetVideoServingEndpoint(message string) error {
return _datastore.SetString(videoServingEndpointKey, message)
}

2
core/data/configEntry.go

@ -8,8 +8,8 @@ import ( @@ -8,8 +8,8 @@ import (
// ConfigEntry is the actual object saved to the database.
// The Value is encoded using encoding/gob.
type ConfigEntry struct {
Value interface{}
Key string
Value interface{}
}
func (c *ConfigEntry) getStringSlice() ([]string, error) {

6
core/data/data_test.go

@ -20,8 +20,6 @@ func TestString(t *testing.T) { @@ -20,8 +20,6 @@ func TestString(t *testing.T) {
const testKey = "test string key"
const testValue = "test string value"
fmt.Println(testKey, testValue)
if err := _datastore.SetString(testKey, testValue); err != nil {
panic(err)
}
@ -89,7 +87,7 @@ func TestCustomType(t *testing.T) { @@ -89,7 +87,7 @@ func TestCustomType(t *testing.T) {
}
// Save config entry to the database
if err := _datastore.Save(ConfigEntry{&testStruct, testKey}); err != nil {
if err := _datastore.Save(ConfigEntry{testKey, &testStruct}); err != nil {
t.Error(err)
}
@ -121,7 +119,7 @@ func TestStringMap(t *testing.T) { @@ -121,7 +119,7 @@ func TestStringMap(t *testing.T) {
}
// Save config entry to the database
if err := _datastore.Save(ConfigEntry{&testMap, testKey}); err != nil {
if err := _datastore.Save(ConfigEntry{testKey, &testMap}); err != nil {
t.Error(err)
}

14
core/data/datastoreMigrations.go

@ -8,7 +8,7 @@ import ( @@ -8,7 +8,7 @@ import (
)
const (
datastoreValuesVersion = 3
datastoreValuesVersion = 2
datastoreValueVersionKey = "DATA_STORE_VERSION"
)
@ -25,8 +25,6 @@ func migrateDatastoreValues(datastore *Datastore) { @@ -25,8 +25,6 @@ func migrateDatastoreValues(datastore *Datastore) {
migrateToDatastoreValues1(datastore)
case 1:
migrateToDatastoreValues2(datastore)
case 2:
migrateToDatastoreValues3ServingEndpoint3(datastore)
default:
log.Fatalln("missing datastore values migration step")
}
@ -63,13 +61,3 @@ func migrateToDatastoreValues2(datastore *Datastore) { @@ -63,13 +61,3 @@ func migrateToDatastoreValues2(datastore *Datastore) {
{Key: oldAdminPassword, Comment: "Default stream key"},
})
}
func migrateToDatastoreValues3ServingEndpoint3(_ *Datastore) {
s3Config := GetS3Config()
if !s3Config.Enabled {
return
}
_ = SetVideoServingEndpoint(s3Config.ServingEndpoint)
}

84
core/data/emoji.go

@ -6,8 +6,6 @@ import ( @@ -6,8 +6,6 @@ import (
"io/fs"
"os"
"path/filepath"
"sync"
"time"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/models"
@ -17,81 +15,29 @@ import ( @@ -17,81 +15,29 @@ import (
log "github.com/sirupsen/logrus"
)
var (
emojiCacheMu sync.Mutex
emojiCacheData = make([]models.CustomEmoji, 0)
emojiCacheModTime time.Time
)
// UpdateEmojiList will update the cache (if required) and
// return the modifiation time.
func UpdateEmojiList(force bool) (time.Time, error) {
var modTime time.Time
emojiPathInfo, err := os.Stat(config.CustomEmojiPath)
if err != nil {
return modTime, err
}
modTime = emojiPathInfo.ModTime()
if modTime.After(emojiCacheModTime) || force {
emojiCacheMu.Lock()
defer emojiCacheMu.Unlock()
// double-check that another thread didn't update this while waiting.
if modTime.After(emojiCacheModTime) || force {
emojiCacheModTime = modTime
if force {
emojiCacheModTime = time.Now()
}
emojiFS := os.DirFS(config.CustomEmojiPath)
if emojiFS == nil {
return modTime, fmt.Errorf("unable to open custom emoji directory")
}
emojiCacheData = make([]models.CustomEmoji, 0)
walkFunction := func(path string, d os.DirEntry, err error) error {
if d == nil || d.IsDir() {
return nil
}
// GetEmojiList returns a list of custom emoji from the emoji directory.
func GetEmojiList() []models.CustomEmoji {
emojiFS := os.DirFS(config.CustomEmojiPath)
emojiPath := filepath.Join(config.EmojiDir, path)
fileName := d.Name()
fileBase := fileName[:len(fileName)-len(filepath.Ext(fileName))]
singleEmoji := models.CustomEmoji{Name: fileBase, URL: emojiPath}
emojiCacheData = append(emojiCacheData, singleEmoji)
return nil
}
emojiResponse := make([]models.CustomEmoji, 0)
if err := fs.WalkDir(emojiFS, ".", walkFunction); err != nil {
log.Errorln("unable to fetch emojis: " + err.Error())
}
walkFunction := func(path string, d os.DirEntry, err error) error {
if d.IsDir() {
return nil
}
}
return modTime, nil
}
// GetEmojiList returns a list of custom emoji from the emoji directory.
func GetEmojiList() []models.CustomEmoji {
_, err := UpdateEmojiList(false)
if err != nil {
emojiPath := filepath.Join(config.EmojiDir, path)
singleEmoji := models.CustomEmoji{Name: d.Name(), URL: emojiPath}
emojiResponse = append(emojiResponse, singleEmoji)
return nil
}
// Lock to make sure this doesn't get updated in the middle of reading
emojiCacheMu.Lock()
defer emojiCacheMu.Unlock()
// return a copy of cache data, ensures underlying slice isn't affected
// by future update
emojiData := make([]models.CustomEmoji, len(emojiCacheData))
copy(emojiData, emojiCacheData)
if err := fs.WalkDir(emojiFS, ".", walkFunction); err != nil {
log.Errorln("unable to fetch emojis: " + err.Error())
return emojiResponse
}
return emojiData
return emojiResponse
}
// SetupEmojiDirectory sets up the custom emoji directory by copying all built-in

4
core/data/migrations.go

@ -283,11 +283,11 @@ func migrateToSchema1(db *sql.DB) { @@ -283,11 +283,11 @@ func migrateToSchema1(db *sql.DB) {
// Migrate access tokens to become chat users
type oldAccessToken struct {
createdAt time.Time
lastUsedAt *time.Time
accessToken string
displayName string
scopes string
createdAt time.Time
lastUsedAt *time.Time
}
oldAccessTokens := make([]oldAccessToken, 0)

10
core/data/types.go

@ -11,7 +11,7 @@ func (ds *Datastore) GetStringSlice(key string) ([]string, error) { @@ -11,7 +11,7 @@ func (ds *Datastore) GetStringSlice(key string) ([]string, error) {
// SetStringSlice will set the string slice value for a key.
func (ds *Datastore) SetStringSlice(key string, value []string) error {
configEntry := ConfigEntry{value, key}
configEntry := ConfigEntry{key, value}
return ds.Save(configEntry)
}
@ -26,7 +26,7 @@ func (ds *Datastore) GetString(key string) (string, error) { @@ -26,7 +26,7 @@ func (ds *Datastore) GetString(key string) (string, error) {
// SetString will set the string value for a key.
func (ds *Datastore) SetString(key string, value string) error {
configEntry := ConfigEntry{value, key}
configEntry := ConfigEntry{key, value}
return ds.Save(configEntry)
}
@ -41,7 +41,7 @@ func (ds *Datastore) GetNumber(key string) (float64, error) { @@ -41,7 +41,7 @@ func (ds *Datastore) GetNumber(key string) (float64, error) {
// SetNumber will set the numeric value for a key.
func (ds *Datastore) SetNumber(key string, value float64) error {
configEntry := ConfigEntry{value, key}
configEntry := ConfigEntry{key, value}
return ds.Save(configEntry)
}
@ -56,7 +56,7 @@ func (ds *Datastore) GetBool(key string) (bool, error) { @@ -56,7 +56,7 @@ func (ds *Datastore) GetBool(key string) (bool, error) {
// SetBool will set the boolean value for a key.
func (ds *Datastore) SetBool(key string, value bool) error {
configEntry := ConfigEntry{value, key}
configEntry := ConfigEntry{key, value}
return ds.Save(configEntry)
}
@ -71,6 +71,6 @@ func (ds *Datastore) GetStringMap(key string) (map[string]string, error) { @@ -71,6 +71,6 @@ func (ds *Datastore) GetStringMap(key string) (map[string]string, error) {
// SetStringMap will set the string map value for a key.
func (ds *Datastore) SetStringMap(key string, value map[string]string) error {
configEntry := ConfigEntry{value, key}
configEntry := ConfigEntry{key, value}
return ds.Save(configEntry)
}

16
core/rtmp/rtmp.go

@ -73,7 +73,7 @@ func HandleConn(c *rtmp.Conn, nc net.Conn) { @@ -73,7 +73,7 @@ func HandleConn(c *rtmp.Conn, nc net.Conn) {
}
if _hasInboundRTMPConnection {
log.Errorln("stream already running; can not overtake an existing stream from", nc.RemoteAddr().String())
log.Errorln("stream already running; can not overtake an existing stream")
_ = nc.Close()
return
}
@ -81,11 +81,6 @@ func HandleConn(c *rtmp.Conn, nc net.Conn) { @@ -81,11 +81,6 @@ func HandleConn(c *rtmp.Conn, nc net.Conn) {
accessGranted := false
validStreamingKeys := data.GetStreamKeys()
// If a stream key override was specified then use that instead.
if config.TemporaryStreamKey != "" {
validStreamingKeys = []models.StreamKey{{Key: config.TemporaryStreamKey}}
}
for _, key := range validStreamingKeys {
if secretMatch(key.Key, c.URL.Path) {
accessGranted = true
@ -93,15 +88,20 @@ func HandleConn(c *rtmp.Conn, nc net.Conn) { @@ -93,15 +88,20 @@ func HandleConn(c *rtmp.Conn, nc net.Conn) {
}
}
// Test against the temporary key if it was set at runtime.
if config.TemporaryStreamKey != "" && secretMatch(config.TemporaryStreamKey, c.URL.Path) {
accessGranted = true
}
if !accessGranted {
log.Errorln("invalid streaming key; rejecting incoming stream from", nc.RemoteAddr().String())
log.Errorln("invalid streaming key; rejecting incoming stream")
_ = nc.Close()
return
}
rtmpOut, rtmpIn := io.Pipe()
_pipe = rtmpIn
log.Infoln("Inbound stream connected from", nc.RemoteAddr().String())
log.Infoln("Inbound stream connected.")
_setStreamAsConnected(rtmpOut)
_hasInboundRTMPConnection = true

95
core/storageproviders/local.go

@ -1,20 +1,18 @@ @@ -1,20 +1,18 @@
package storageproviders
import (
"os"
"path/filepath"
"sort"
"time"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/core/transcoder"
)
// LocalStorage represents an instance of the local storage provider for HLS video.
type LocalStorage struct {
host string
// Cleanup old public HLS content every N min from the webroot.
onlineCleanupTicker *time.Ticker
}
// NewLocalStorage returns a new LocalStorage instance.
@ -24,7 +22,14 @@ func NewLocalStorage() *LocalStorage { @@ -24,7 +22,14 @@ func NewLocalStorage() *LocalStorage {
// Setup configures this storage provider.
func (s *LocalStorage) Setup() error {
s.host = data.GetVideoServingEndpoint()
// NOTE: This cleanup timer will have to be disabled to support recordings in the future
// as all HLS segments have to be publicly available on disk to keep a recording of them.
s.onlineCleanupTicker = time.NewTicker(1 * time.Minute)
go func() {
for range s.onlineCleanupTicker.C {
transcoder.CleanupOldContent(config.HLSStoragePath)
}
}()
return nil
}
@ -45,15 +50,8 @@ func (s *LocalStorage) VariantPlaylistWritten(localFilePath string) { @@ -45,15 +50,8 @@ func (s *LocalStorage) VariantPlaylistWritten(localFilePath string) {
// MasterPlaylistWritten is called when the master hls playlist is written.
func (s *LocalStorage) MasterPlaylistWritten(localFilePath string) {
// If we're using a remote serving endpoint, we need to rewrite the master playlist
if s.host != "" {
if err := rewritePlaylistLocations(localFilePath, s.host, ""); err != nil {
log.Warnln(err)
}
} else {
if _, err := s.Save(localFilePath, 0); err != nil {
log.Warnln(err)
}
if _, err := s.Save(localFilePath, 0); err != nil {
log.Warnln(err)
}
}
@ -61,68 +59,3 @@ func (s *LocalStorage) MasterPlaylistWritten(localFilePath string) { @@ -61,68 +59,3 @@ func (s *LocalStorage) MasterPlaylistWritten(localFilePath string) {
func (s *LocalStorage) Save(filePath string, retryCount int) (string, error) {
return filePath, nil
}
func (s *LocalStorage) Cleanup() error {
// Determine how many files we should keep on disk
maxNumber := data.GetStreamLatencyLevel().SegmentCount
buffer := 10
baseDirectory := config.HLSStoragePath
files, err := getAllFilesRecursive(baseDirectory)
if err != nil {
return errors.Wrap(err, "unable find old video files for cleanup")
}
// Delete old private HLS files on disk
for directory := range files {
files := files[directory]
if len(files) < maxNumber+buffer {
continue
}
filesToDelete := files[maxNumber+buffer:]
log.Traceln("Deleting", len(filesToDelete), "old files from", baseDirectory, "for video variant", directory)
for _, file := range filesToDelete {
fileToDelete := filepath.Join(baseDirectory, directory, file.Name())
err := os.Remove(fileToDelete)
if err != nil {
return errors.Wrap(err, "unable to delete old video files")
}
}
}
return nil
}
func getAllFilesRecursive(baseDirectory string) (map[string][]os.FileInfo, error) {
files := make(map[string][]os.FileInfo)
var directory string
err := filepath.Walk(baseDirectory, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
directory = info.Name()
}
if filepath.Ext(info.Name()) == ".ts" {
files[directory] = append(files[directory], info)
}
return nil
})
if err != nil {
return nil, err
}
// Sort by date so we can delete old files
for directory := range files {
sort.Slice(files[directory], func(i, j int) bool {
return files[directory][i].ModTime().UnixNano() > files[directory][j].ModTime().UnixNano()
})
}
return files, nil
}

43
core/storageproviders/rewriteLocalPlaylist.go

@ -1,43 +0,0 @@ @@ -1,43 +0,0 @@
package storageproviders
import (
"bufio"
"os"
"path/filepath"
"github.com/grafov/m3u8"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core/playlist"
log "github.com/sirupsen/logrus"
)
// rewritePlaylistLocations will take a local playlist and rewrite it to have absolute URLs to a specified location.
func rewritePlaylistLocations(localFilePath, remoteServingEndpoint, pathPrefix string) error {
f, err := os.Open(localFilePath) // nolint
if err != nil {
log.Fatalln(err)
}
p := m3u8.NewMasterPlaylist()
if err := p.DecodeFrom(bufio.NewReader(f), false); err != nil {
log.Warnln(err)
}
for _, item := range p.Variants {
// Determine the final path to this playlist.
var finalPath string
if pathPrefix != "" {
finalPath = filepath.Join(pathPrefix, "/hls")
} else {
finalPath = "/hls"
}
item.URI = remoteServingEndpoint + filepath.Join(finalPath, item.URI)
}
publicPath := filepath.Join(config.HLSStoragePath, filepath.Base(localFilePath))
newPlaylist := p.String()
return playlist.WritePlaylist(newPlaylist, publicPath)
}

175
core/storageproviders/s3Storage.go

@ -1,63 +1,54 @@ @@ -1,63 +1,54 @@
package storageproviders
import (
"bufio"
"fmt"
"net/http"
"os"
"path"
"path/filepath"
"sort"
"strings"
"sync"
"time"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/core/playlist"
"github.com/owncast/owncast/utils"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/aws/aws-sdk-go/service/s3/s3manager"
"github.com/owncast/owncast/config"
"github.com/grafov/m3u8"
)
// S3Storage is the s3 implementation of a storage provider.
type S3Storage struct {
// If we try to upload a playlist but it is not yet on disk
// then keep a reference to it here.
queuedPlaylistUpdates map[string]string
s3Client *s3.S3
uploader *s3manager.Uploader
sess *session.Session
s3Secret string
sess *session.Session
host string
s3Bucket string
s3Region string
s3Endpoint string
s3ServingEndpoint string
s3Region string
s3Bucket string
s3AccessKey string
s3Secret string
s3ACL string
s3PathPrefix string
s3ForcePathStyle bool
s3Endpoint string
host string
lock sync.Mutex
// If we try to upload a playlist but it is not yet on disk
// then keep a reference to it here.
queuedPlaylistUpdates map[string]string
s3ForcePathStyle bool
uploader *s3manager.Uploader
}
// NewS3Storage returns a new S3Storage instance.
func NewS3Storage() *S3Storage {
return &S3Storage{
queuedPlaylistUpdates: make(map[string]string),
lock: sync.Mutex{},
}
}
@ -66,10 +57,8 @@ func (s *S3Storage) Setup() error { @@ -66,10 +57,8 @@ func (s *S3Storage) Setup() error {
log.Trace("Setting up S3 for external storage of video...")
s3Config := data.GetS3Config()
customVideoServingEndpoint := data.GetVideoServingEndpoint()
if customVideoServingEndpoint != "" {
s.host = customVideoServingEndpoint
if s3Config.ServingEndpoint != "" {
s.host = s3Config.ServingEndpoint
} else {
s.host = fmt.Sprintf("%s/%s", s3Config.Endpoint, s3Config.Bucket)
}
@ -81,11 +70,9 @@ func (s *S3Storage) Setup() error { @@ -81,11 +70,9 @@ func (s *S3Storage) Setup() error {
s.s3AccessKey = s3Config.AccessKey
s.s3Secret = s3Config.Secret
s.s3ACL = s3Config.ACL
s.s3PathPrefix = s3Config.PathPrefix
s.s3ForcePathStyle = s3Config.ForcePathStyle
s.sess = s.connectAWS()
s.s3Client = s3.New(s.sess)
s.uploader = s3manager.NewUploader(s.sess)
@ -116,7 +103,6 @@ func (s *S3Storage) SegmentWritten(localFilePath string) { @@ -116,7 +103,6 @@ func (s *S3Storage) SegmentWritten(localFilePath string) {
// so the segments and the HLS playlist referencing
// them are in sync.
playlistPath := filepath.Join(filepath.Dir(localFilePath), "stream.m3u8")
if _, err := s.Save(playlistPath, 0); err != nil {
s.queuedPlaylistUpdates[playlistPath] = playlistPath
if pErr, ok := err.(*os.PathError); ok {
@ -131,8 +117,6 @@ func (s *S3Storage) VariantPlaylistWritten(localFilePath string) { @@ -131,8 +117,6 @@ func (s *S3Storage) VariantPlaylistWritten(localFilePath string) {
// We are uploading the variant playlist after uploading the segment
// to make sure we're not referring to files in a playlist that don't
// yet exist. See SegmentWritten.
s.lock.Lock()
defer s.lock.Unlock()
if _, ok := s.queuedPlaylistUpdates[localFilePath]; ok {
if _, err := s.Save(localFilePath, 0); err != nil {
log.Errorln(err)
@ -145,7 +129,7 @@ func (s *S3Storage) VariantPlaylistWritten(localFilePath string) { @@ -145,7 +129,7 @@ func (s *S3Storage) VariantPlaylistWritten(localFilePath string) {
// MasterPlaylistWritten is called when the master hls playlist is written.
func (s *S3Storage) MasterPlaylistWritten(localFilePath string) {
// Rewrite the playlist to use absolute remote S3 URLs
if err := rewritePlaylistLocations(localFilePath, s.host, s.s3PathPrefix); err != nil {
if err := s.rewriteRemotePlaylist(localFilePath); err != nil {
log.Warnln(err)
}
}
@ -163,15 +147,8 @@ func (s *S3Storage) Save(filePath string, retryCount int) (string, error) { @@ -163,15 +147,8 @@ func (s *S3Storage) Save(filePath string, retryCount int) (string, error) {
// Build the remote path by adding the "hls" path prefix.
remotePath := strings.Join([]string{"hls", normalizedPath}, "")
// If a custom path prefix is set prepend it.
if s.s3PathPrefix != "" {
prefix := strings.TrimPrefix(s.s3PathPrefix, "/")
remotePath = strings.Join([]string{prefix, remotePath}, "/")
}
maxAgeSeconds := utils.GetCacheDurationSecondsForPath(filePath)
cacheControlHeader := fmt.Sprintf("max-age=%d", maxAgeSeconds)
uploadInput := &s3manager.UploadInput{
Bucket: aws.String(s.s3Bucket), // Bucket to be used
Key: aws.String(remotePath), // Name of the file to be saved
@ -179,14 +156,6 @@ func (s *S3Storage) Save(filePath string, retryCount int) (string, error) { @@ -179,14 +156,6 @@ func (s *S3Storage) Save(filePath string, retryCount int) (string, error) {
CacheControl: &cacheControlHeader,
}
if path.Ext(filePath) == ".m3u8" {
noCacheHeader := "no-cache, no-store, must-revalidate"
contentType := "application/x-mpegURL"
uploadInput.CacheControl = &noCacheHeader
uploadInput.ContentType = &contentType
}
if s.s3ACL != "" {
uploadInput.ACL = aws.String(s.s3ACL)
} else {
@ -202,35 +171,12 @@ func (s *S3Storage) Save(filePath string, retryCount int) (string, error) { @@ -202,35 +171,12 @@ func (s *S3Storage) Save(filePath string, retryCount int) (string, error) {
return s.Save(filePath, retryCount+1)
}
// Upload failure. Remove the local file.
s.removeLocalFile(filePath)
return "", fmt.Errorf("Giving up uploading %s to object storage %s", filePath, s.s3Endpoint)
return "", fmt.Errorf("Giving up on %s", filePath)
}
// Upload success. Remove the local file.
s.removeLocalFile(filePath)
return response.Location, nil
}
func (s *S3Storage) Cleanup() error {
// Determine how many files we should keep on S3 storage
maxNumber := data.GetStreamLatencyLevel().SegmentCount
buffer := 20
keys, err := s.getDeletableVideoSegmentsWithOffset(maxNumber + buffer)
if err != nil {
return err
}
if len(keys) > 0 {
s.deleteObjects(keys)
}
return nil
}
func (s *S3Storage) connectAWS() *session.Session {
t := http.DefaultTransport.(*http.Transport).Clone()
t.MaxIdleConnsPerHost = 100
@ -261,84 +207,25 @@ func (s *S3Storage) connectAWS() *session.Session { @@ -261,84 +207,25 @@ func (s *S3Storage) connectAWS() *session.Session {
return sess
}
func (s *S3Storage) getDeletableVideoSegmentsWithOffset(offset int) ([]s3object, error) {
objectsToDelete, err := s.retrieveAllVideoSegments()
// rewriteRemotePlaylist will take a local playlist and rewrite it to have absolute URLs to remote locations.
func (s *S3Storage) rewriteRemotePlaylist(filePath string) error {
f, err := os.Open(filePath) // nolint
if err != nil {
return nil, err
}
if offset > len(objectsToDelete)-1 {
offset = len(objectsToDelete) - 1
}
objectsToDelete = objectsToDelete[offset : len(objectsToDelete)-1]
return objectsToDelete, nil
}
func (s *S3Storage) removeLocalFile(filePath string) {
cleanFilepath := filepath.Clean(filePath)
if err := os.Remove(cleanFilepath); err != nil {
log.Errorln(err)
}
}
func (s *S3Storage) deleteObjects(objects []s3object) {
keys := make([]*s3.ObjectIdentifier, len(objects))
for i, object := range objects {
keys[i] = &s3.ObjectIdentifier{Key: aws.String(object.key)}
log.Fatalln(err)
}
log.Debugln("Deleting", len(keys), "objects from S3 bucket:", s.s3Bucket)
deleteObjectsRequest := &s3.DeleteObjectsInput{
Bucket: aws.String(s.s3Bucket),
Delete: &s3.Delete{
Objects: keys,
Quiet: aws.Bool(true),
},
}
_, err := s.s3Client.DeleteObjects(deleteObjectsRequest)
if err != nil {
log.Errorf("Unable to delete objects from bucket %q, %v\n", s.s3Bucket, err)
}
}
func (s *S3Storage) retrieveAllVideoSegments() ([]s3object, error) {
allObjectsListRequest := &s3.ListObjectsInput{
Bucket: aws.String(s.s3Bucket),
p := m3u8.NewMasterPlaylist()
if err := p.DecodeFrom(bufio.NewReader(f), false); err != nil {
log.Warnln(err)
}
// Fetch all objects in the bucket
allObjectsListResponse, err := s.s3Client.ListObjects(allObjectsListRequest)
if err != nil {
return nil, errors.Wrap(err, "Unable to fetch list of items in bucket for cleanup")
for _, item := range p.Variants {
item.URI = s.host + filepath.Join("/hls", item.URI)
}
// Filter out non-video segments
allObjects := []s3object{}
for _, item := range allObjectsListResponse.Contents {
if !strings.HasSuffix(*item.Key, ".ts") {
continue
}
publicPath := filepath.Join(config.HLSStoragePath, filepath.Base(filePath))
allObjects = append(allObjects, s3object{
key: *item.Key,
lastModified: *item.LastModified,
})
}
// Sort the results by timestamp
sort.Slice(allObjects, func(i, j int) bool {
return allObjects[i].lastModified.After(allObjects[j].lastModified)
})
return allObjects, nil
}
newPlaylist := p.String()
type s3object struct {
lastModified time.Time
key string
return playlist.WritePlaylist(newPlaylist, publicPath)
}

9
core/streamState.go

@ -65,12 +65,11 @@ func setStreamAsConnected(rtmpOut *io.PipeReader) { @@ -65,12 +65,11 @@ func setStreamAsConnected(rtmpOut *io.PipeReader) {
_currentBroadcast = nil
}
_transcoder.SetStdin(rtmpOut)
_transcoder.Start(true)
_transcoder.Start()
}()
go webhooks.SendStreamStatusEvent(models.StreamStarted)
selectedThumbnailVideoQualityIndex, isVideoPassthrough := data.FindHighestVideoQualityIndex(_currentBroadcast.OutputSettings)
transcoder.StartThumbnailGenerator(segmentPath, selectedThumbnailVideoQualityIndex, isVideoPassthrough)
transcoder.StartThumbnailGenerator(segmentPath, data.FindHighestVideoQualityIndex(_currentBroadcast.OutputSettings))
_ = chat.SendSystemAction("Stay tuned, the stream is **starting**!", true)
chat.SendAllWelcomeMessage()
@ -152,9 +151,7 @@ func startOnlineCleanupTimer() { @@ -152,9 +151,7 @@ func startOnlineCleanupTimer() {
_onlineCleanupTicker = time.NewTicker(1 * time.Minute)
go func() {
for range _onlineCleanupTicker.C {
if err := _storage.Cleanup(); err != nil {
log.Errorln(err)
}
transcoder.CleanupOldContent(config.HLSStoragePath)
}
}()
}

42
core/transcoder/codecs.go

@ -16,7 +16,6 @@ type Codec interface { @@ -16,7 +16,6 @@ type Codec interface {
DisplayName() string
GlobalFlags() string
PixelFormat() string
Scaler() string
ExtraArguments() string
ExtraFilters() string
VariantFlags(v *HLSVariant) string
@ -54,11 +53,6 @@ func (c *Libx264Codec) PixelFormat() string { @@ -54,11 +53,6 @@ func (c *Libx264Codec) PixelFormat() string {
return "yuv420p" //nolint:goconst
}
// Scaler is the scaler used for resizing the video in the transcoder.
func (c *Libx264Codec) Scaler() string {
return ""
}
// ExtraArguments are the extra arguments used with this codec in the transcoder.
func (c *Libx264Codec) ExtraArguments() string {
return strings.Join([]string{
@ -123,11 +117,6 @@ func (c *OmxCodec) PixelFormat() string { @@ -123,11 +117,6 @@ func (c *OmxCodec) PixelFormat() string {
return "yuv420p"
}
// Scaler is the scaler used for resizing the video in the transcoder.
func (c *OmxCodec) Scaler() string {
return ""
}
// ExtraArguments are the extra arguments used with this codec in the transcoder.
func (c *OmxCodec) ExtraArguments() string {
return strings.Join([]string{
@ -181,8 +170,6 @@ func (c *VaapiCodec) DisplayName() string { @@ -181,8 +170,6 @@ func (c *VaapiCodec) DisplayName() string {
// GlobalFlags are the global flags used with this codec in the transcoder.
func (c *VaapiCodec) GlobalFlags() string {
flags := []string{
"-hwaccel", "vaapi",
"-hwaccel_output_format", "vaapi",
"-vaapi_device", "/dev/dri/renderD128",
}
@ -194,14 +181,9 @@ func (c *VaapiCodec) PixelFormat() string { @@ -194,14 +181,9 @@ func (c *VaapiCodec) PixelFormat() string {
return "vaapi_vld"
}
// Scaler is the scaler used for resizing the video in the transcoder.
func (c *VaapiCodec) Scaler() string {
return "scale_vaapi"
}
// ExtraFilters are the extra filters required for this codec in the transcoder.
func (c *VaapiCodec) ExtraFilters() string {
return ""
return "format=nv12,hwupload"
}
// ExtraArguments are the extra arguments used with this codec in the transcoder.
@ -250,7 +232,7 @@ func (c *NvencCodec) DisplayName() string { @@ -250,7 +232,7 @@ func (c *NvencCodec) DisplayName() string {
// GlobalFlags are the global flags used with this codec in the transcoder.
func (c *NvencCodec) GlobalFlags() string {
flags := []string{
"-hwaccel", "cuda",
"-hwaccel cuda",
}
return strings.Join(flags, " ")
@ -261,11 +243,6 @@ func (c *NvencCodec) PixelFormat() string { @@ -261,11 +243,6 @@ func (c *NvencCodec) PixelFormat() string {
return "yuv420p"
}
// Scaler is the scaler used for resizing the video in the transcoder.
func (c *NvencCodec) Scaler() string {
return ""
}
// ExtraArguments are the extra arguments used with this codec in the transcoder.
func (c *NvencCodec) ExtraArguments() string {
return ""
@ -325,11 +302,6 @@ func (c *QuicksyncCodec) PixelFormat() string { @@ -325,11 +302,6 @@ func (c *QuicksyncCodec) PixelFormat() string {
return "nv12"
}
// Scaler is the scaler used for resizing the video in the transcoder.
func (c *QuicksyncCodec) Scaler() string {
return ""
}
// ExtraArguments are the extra arguments used with this codec in the transcoder.
func (c *QuicksyncCodec) ExtraArguments() string {
return ""
@ -388,11 +360,6 @@ func (c *Video4Linux) PixelFormat() string { @@ -388,11 +360,6 @@ func (c *Video4Linux) PixelFormat() string {
return "nv21"
}
// Scaler is the scaler used for resizing the video in the transcoder.
func (c *Video4Linux) Scaler() string {
return ""
}
// ExtraArguments are the extra arguments used with this codec in the transcoder.
func (c *Video4Linux) ExtraArguments() string {
return ""
@ -452,11 +419,6 @@ func (c *VideoToolboxCodec) PixelFormat() string { @@ -452,11 +419,6 @@ func (c *VideoToolboxCodec) PixelFormat() string {
return "nv12"
}
// Scaler is the scaler used for resizing the video in the transcoder.
func (c *VideoToolboxCodec) Scaler() string {
return ""
}
// ExtraFilters are the extra filters required for this codec in the transcoder.
func (c *VideoToolboxCodec) ExtraFilters() string {
return ""

3
core/transcoder/fileWriterReceiverService.go

@ -21,9 +21,6 @@ type FileWriterReceiverServiceCallback interface { @@ -21,9 +21,6 @@ type FileWriterReceiverServiceCallback interface {
}
// FileWriterReceiverService accepts transcoder responses via HTTP and fires the callbacks.
// It is intended to be the middleman between the transcoder and the storage provider and allows
// the transcoder process to be completely isolated and even run remotely in the future, as long
// as it can send HTTP requests to this service with the results.
type FileWriterReceiverService struct {
callbacks FileWriterReceiverServiceCallback
}

78
core/transcoder/hlsFilesystemCleanup.go

@ -0,0 +1,78 @@ @@ -0,0 +1,78 @@
package transcoder
import (
log "github.com/sirupsen/logrus"
"os"
"path/filepath"
"sort"
"github.com/owncast/owncast/core/data"
)
// CleanupOldContent will delete old files from the private dir that are no longer being referenced
// in the stream.
func CleanupOldContent(baseDirectory string) {
// Determine how many files we should keep on disk
maxNumber := data.GetStreamLatencyLevel().SegmentCount
buffer := 10
files, err := getAllFilesRecursive(baseDirectory)
if err != nil {
log.Debugln("Unable to cleanup old video files", err)
return
}
// Delete old private HLS files on disk
for directory := range files {
files := files[directory]
if len(files) < maxNumber+buffer {
continue
}
filesToDelete := files[maxNumber+buffer:]
log.Traceln("Deleting", len(filesToDelete), "old files from", baseDirectory, "for video variant", directory)
for _, file := range filesToDelete {
fileToDelete := filepath.Join(baseDirectory, directory, file.Name())
err := os.Remove(fileToDelete)
if err != nil {
log.Debugln(err)
}
}
}
}
func getAllFilesRecursive(baseDirectory string) (map[string][]os.FileInfo, error) {
var files = make(map[string][]os.FileInfo)
var directory string
err := filepath.Walk(baseDirectory, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
directory = info.Name()
}
if filepath.Ext(info.Name()) == ".ts" {
files[directory] = append(files[directory], info)
}
return nil
})
if err != nil {
return nil, err
}
// Sort by date so we can delete old files
for directory := range files {
sort.Slice(files[directory], func(i, j int) bool {
return files[directory][i].ModTime().UnixNano() > files[directory][j].ModTime().UnixNano()
})
}
return files, nil
}

8
core/transcoder/thumbnailGenerator.go

@ -25,7 +25,7 @@ func StopThumbnailGenerator() { @@ -25,7 +25,7 @@ func StopThumbnailGenerator() {
}
// StartThumbnailGenerator starts generating thumbnails.
func StartThumbnailGenerator(chunkPath string, variantIndex int, isVideoPassthrough bool) {
func StartThumbnailGenerator(chunkPath string, variantIndex int) {
// Every 20 seconds create a thumbnail from the most
// recent video segment.
_timer = time.NewTicker(20 * time.Second)
@ -36,11 +36,7 @@ func StartThumbnailGenerator(chunkPath string, variantIndex int, isVideoPassthro @@ -36,11 +36,7 @@ func StartThumbnailGenerator(chunkPath string, variantIndex int, isVideoPassthro
select {
case <-_timer.C:
if err := fireThumbnailGenerator(chunkPath, variantIndex); err != nil {
logMsg := "Unable to generate thumbnail: " + err.Error()
if isVideoPassthrough {
logMsg += ". Video Passthrough is enabled. You should disable it to fix this, and other, streaming errors. https://owncast.online/troubleshoot"
}
log.Errorln("Unable to generate thumbnail:", logMsg)
log.Errorln("Unable to generate thumbnail:", err)
}
case <-quit:
log.Debug("thumbnail generator has stopped")

50
core/transcoder/transcoder.go

@ -22,39 +22,37 @@ var _commandExec *exec.Cmd @@ -22,39 +22,37 @@ var _commandExec *exec.Cmd
// Transcoder is a single instance of a video transcoder.
type Transcoder struct {
codec Codec
stdin *io.PipeReader
TranscoderCompleted func(error)
input string
stdin *io.PipeReader
segmentOutputPath string
playlistOutputPath string
variants []HLSVariant
appendToStream bool
ffmpegPath string
segmentIdentifier string
internalListenerPort string
input string
segmentOutputPath string
variants []HLSVariant
codec Codec
currentStreamOutputSettings []models.StreamOutputVariant
currentLatencyLevel models.LatencyLevel
appendToStream bool
isEvent bool
TranscoderCompleted func(error)
}
// HLSVariant is a combination of settings that results in a single HLS stream.
type HLSVariant struct {
audioBitrate string // The audio bitrate
index int
videoSize VideoSize // Resizes the video via scaling
index int
videoSize VideoSize // Resizes the video via scaling
framerate int // The output framerate
videoBitrate int // The output bitrate
isVideoPassthrough bool // Override all settings and just copy the video stream
framerate int // The output framerate
videoBitrate int // The output bitrate
audioBitrate string // The audio bitrate
isAudioPassthrough bool // Override all settings and just copy the audio stream
cpuUsageLevel int // The amount of hardware to use for encoding a stream
isVideoPassthrough bool // Override all settings and just copy the video stream
isAudioPassthrough bool // Override all settings and just copy the audio stream
cpuUsageLevel int // The amount of hardware to use for encoding a stream
}
// VideoSize is the scaled size of the video output.
@ -111,13 +109,11 @@ func (t *Transcoder) Stop() { @@ -111,13 +109,11 @@ func (t *Transcoder) Stop() {
}
// Start will execute the transcoding process with the settings previously set.
func (t *Transcoder) Start(shouldLog bool) {
func (t *Transcoder) Start() {
_lastTranscoderLogMessage = ""
command := t.getString()
if shouldLog {
log.Infof("Processing video using codec %s with %d output qualities configured.", t.codec.DisplayName(), len(t.variants))
}
log.Infof("Video transcoder started using %s with %d stream variants.", t.codec.DisplayName(), len(t.variants))
createVariantDirectories()
if config.EnableDebugFeatures {
@ -204,7 +200,6 @@ func (t *Transcoder) getString() string { @@ -204,7 +200,6 @@ func (t *Transcoder) getString() string {
"-loglevel warning",
t.codec.GlobalFlags(),
"-fflags +genpts", // Generate presentation time stamp if missing
"-flags +cgop", // Force closed GOPs
"-i ", t.input,
t.getVariantsString(),
@ -305,7 +300,7 @@ func (v *HLSVariant) getVariantString(t *Transcoder) string { @@ -305,7 +300,7 @@ func (v *HLSVariant) getVariantString(t *Transcoder) string {
if (v.videoSize.Width != 0 || v.videoSize.Height != 0) && !v.isVideoPassthrough {
// Order here matters, you must scale before changing hardware formats
filters := []string{
v.getScalingString(t.codec.Scaler()),
v.getScalingString(),
}
if t.codec.ExtraFilters() != "" {
filters = append(filters, t.codec.ExtraFilters())
@ -356,11 +351,8 @@ func (v *HLSVariant) SetVideoScalingHeight(height int) { @@ -356,11 +351,8 @@ func (v *HLSVariant) SetVideoScalingHeight(height int) {
v.videoSize.Height = height
}
func (v *HLSVariant) getScalingString(scaler string) string {
if scaler == "" {
scaler = "scale"
}
return fmt.Sprintf("%s=%s", scaler, v.videoSize.getString())
func (v *HLSVariant) getScalingString() string {
return fmt.Sprintf("scale=%s", v.videoSize.getString())
}
// Video Quality

2
core/transcoder/transcoder_nvenc_test.go

@ -42,7 +42,7 @@ func TestFFmpegNvencCommand(t *testing.T) { @@ -42,7 +42,7 @@ func TestFFmpegNvencCommand(t *testing.T) {
cmd := transcoder.getString()
expectedLogPath := filepath.Join("data", "logs", "transcoder.log")
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -hwaccel cuda -fflags +genpts -flags +cgop -i fakecontent.flv -map v:0 -c:v:0 h264_nvenc -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -tune:v:0 ll -map a:0? -c:a:0 copy -preset p3 -map v:0 -c:v:1 h264_nvenc -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -tune:v:1 ll -map a:0? -c:a:1 copy -preset p5 -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset p1 -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -pix_fmt yuv420p -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdoieGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -hwaccel cuda -fflags +genpts -i fakecontent.flv -map v:0 -c:v:0 h264_nvenc -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -tune:v:0 ll -map a:0? -c:a:0 copy -preset p3 -map v:0 -c:v:1 h264_nvenc -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -tune:v:1 ll -map a:0? -c:a:1 copy -preset p5 -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset p1 -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -pix_fmt yuv420p -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdoieGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
if cmd != expected {
t.Errorf("ffmpeg command does not match expected.\nGot %s\n, want: %s", cmd, expected)

2
core/transcoder/transcoder_omx_test.go

@ -42,7 +42,7 @@ func TestFFmpegOmxCommand(t *testing.T) { @@ -42,7 +42,7 @@ func TestFFmpegOmxCommand(t *testing.T) {
cmd := transcoder.getString()
expectedLogPath := filepath.Join("data", "logs", "transcoder.log")
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -fflags +genpts -flags +cgop -i fakecontent.flv -map v:0 -c:v:0 h264_omx -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -map a:0? -c:a:0 copy -preset veryfast -map v:0 -c:v:1 h264_omx -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -map a:0? -c:a:1 copy -preset fast -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset ultrafast -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -tune zerolatency -pix_fmt yuv420p -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdFsdfzGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -fflags +genpts -i fakecontent.flv -map v:0 -c:v:0 h264_omx -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -map a:0? -c:a:0 copy -preset veryfast -map v:0 -c:v:1 h264_omx -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -map a:0? -c:a:1 copy -preset fast -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset ultrafast -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -tune zerolatency -pix_fmt yuv420p -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdFsdfzGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
if cmd != expected {
t.Errorf("ffmpeg command does not match expected.\nGot %s\n, want: %s", cmd, expected)

2
core/transcoder/transcoder_vaapi_test.go

@ -42,7 +42,7 @@ func TestFFmpegVaapiCommand(t *testing.T) { @@ -42,7 +42,7 @@ func TestFFmpegVaapiCommand(t *testing.T) {
cmd := transcoder.getString()
expectedLogPath := filepath.Join("data", "logs", "transcoder.log")
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_output_format vaapi -vaapi_device /dev/dri/renderD128 -fflags +genpts -flags +cgop -i fakecontent.flv -map v:0 -c:v:0 h264_vaapi -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -map a:0? -c:a:0 copy -preset veryfast -map v:0 -c:v:1 h264_vaapi -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -map a:0? -c:a:1 copy -preset fast -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset ultrafast -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -pix_fmt vaapi_vld -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdofFGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -vaapi_device /dev/dri/renderD128 -fflags +genpts -i fakecontent.flv -map v:0 -c:v:0 h264_vaapi -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -map a:0? -c:a:0 copy -filter:v:0 "format=nv12,hwupload" -preset veryfast -map v:0 -c:v:1 h264_vaapi -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -map a:0? -c:a:1 copy -filter:v:1 "format=nv12,hwupload" -preset fast -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset ultrafast -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -pix_fmt vaapi_vld -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdofFGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
if cmd != expected {
t.Errorf("ffmpeg command does not match expected.\nGot %s\n, want: %s", cmd, expected)

2
core/transcoder/transcoder_videotoolbox_test.go

@ -42,7 +42,7 @@ func TestFFmpegVideoToolboxCommand(t *testing.T) { @@ -42,7 +42,7 @@ func TestFFmpegVideoToolboxCommand(t *testing.T) {
cmd := transcoder.getString()
expectedLogPath := filepath.Join("data", "logs", "transcoder.log")
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -fflags +genpts -flags +cgop -i fakecontent.flv -map v:0 -c:v:0 h264_videotoolbox -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -realtime true -map a:0? -c:a:0 copy -preset veryfast -map v:0 -c:v:1 h264_videotoolbox -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -map a:0? -c:a:1 copy -preset fast -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset ultrafast -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -pix_fmt nv12 -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdFsdfzGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
expected := `FFREPORT=file="` + expectedLogPath + `":level=32 ` + transcoder.ffmpegPath + ` -hide_banner -loglevel warning -fflags +genpts -i fakecontent.flv -map v:0 -c:v:0 h264_videotoolbox -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 90 -keyint_min:v:0 90 -r:v:0 30 -realtime true -map a:0? -c:a:0 copy -preset veryfast -map v:0 -c:v:1 h264_videotoolbox -b:v:1 3308k -maxrate:v:1 3572k -g:v:1 72 -keyint_min:v:1 72 -r:v:1 24 -map a:0? -c:a:1 copy -preset fast -map v:0 -c:v:2 copy -map a:0? -c:a:2 copy -preset ultrafast -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options mpegts_flags=mpegts_copyts=1 -pix_fmt nv12 -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename http://127.0.0.1:8123/%v/stream-jdFsdfzGg-%d.ts -max_muxing_queue_size 400 -method PUT http://127.0.0.1:8123/%v/stream.m3u8`
if cmd != expected {
t.Errorf("ffmpeg command does not match expected.\nGot %s\n, want: %s", cmd, expected)

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save