Nulo/forgejo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update queue workers for v1.15 (#15999)

Browse source 
* Update queue workers for v1.15

* update app.example.ini

* update re queue path

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
This commit is contained in:
Kyle D 2021-05-28 11:33:52 -06:00 committed by GitHub
parent e39e79d3ff
commit a5d8f58341
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 9 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
custom/conf/app.example.ini
View file

@ -1145,8 +1145,8 @@ PATH =
;;
;; When ISSUE_INDEXER_QUEUE_TYPE is levelqueue, this will be the path where the queue will be saved.
;; This can be overridden by `ISSUE_INDEXER_QUEUE_CONN_STR`.
;; default is indexers/issues.queue
;ISSUE_INDEXER_QUEUE_DIR = indexers/issues.queue
;; default is queues/common
;ISSUE_INDEXER_QUEUE_DIR = queues/common
;;
;; When `ISSUE_INDEXER_QUEUE_TYPE` is `redis`, this will store the redis connection string.
;; When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this is a directory or additional options of
@ -1201,7 +1201,7 @@ PATH =
;; default to persistable-channel
;TYPE = persistable-channel
;;
;; data-dir for storing persistable queues and level queues, individual queues will be named by their type
;; data-dir for storing persistable queues and level queues, individual queues will default to `queues/common` meaning the queue is shared.
;DATADIR = queues/
;;
;; Default queue length before a channel queue will block
@ -1231,7 +1231,7 @@ PATH =
;TIMEOUT = 15m30s
;;
;; Create a pool with this many workers
;WORKERS = 1
;WORKERS = 0
;;
;; Dynamically scale the worker pool to at this many workers
;MAX_WORKERS = 10
@ -1243,7 +1243,7 @@ PATH =
;BOOST_TIMEOUT = 5m
;;
;; During a boost add BOOST_WORKERS
;BOOST_WORKERS = 5
;BOOST_WORKERS = 1
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

8
docs/content/doc/advanced/config-cheat-sheet.en-us.md
View file

@ -350,7 +350,7 @@ relation to port exhaustion.
- `ISSUE_INDEXER_PATH`: **indexers/issues.bleve**: Index file used for issue search; available when ISSUE_INDEXER_TYPE is bleve and elasticsearch.
- The next 4 configuration values are deprecated and should be set in `queue.issue_indexer` however are kept for backwards compatibility:
- `ISSUE_INDEXER_QUEUE_TYPE`: **levelqueue**: Issue indexer queue, currently supports:`channel`, `levelqueue`, `redis`.
- `ISSUE_INDEXER_QUEUE_DIR`: **indexers/issues.queue**: When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this will be the path where the queue will be saved.
- `ISSUE_INDEXER_QUEUE_DIR`: **queues/common**: When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this will be the path where the queue will be saved. (Previously this was `indexers/issues.queue`.)
- `ISSUE_INDEXER_QUEUE_CONN_STR`: **addrs=127.0.0.1:6379 db=0**: When `ISSUE_INDEXER_QUEUE_TYPE` is `redis`, this will store the redis connection string. When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this is a directory or additional options of the form `leveldb://path/to/db?option=value&....`, and overrides `ISSUE_INDEXER_QUEUE_DIR`.
- `ISSUE_INDEXER_QUEUE_BATCH_NUMBER`: **20**: Batch queue number.
@ -370,7 +370,7 @@ relation to port exhaustion.
## Queue (`queue` and `queue.*`)
- `TYPE`: **persistable-channel**: General queue type, currently support: `persistable-channel` (uses a LevelDB internally), `channel`, `level`, `redis`, `dummy`
- `DATADIR`: **queues/**: Base DataDir for storing persistent and level queues. `DATADIR` for individual queues can be set in `queue.name` sections but will default to `DATADIR/`**`name`**.
- `DATADIR`: **queues/**: Base DataDir for storing persistent and level queues. `DATADIR` for individual queues can be set in `queue.name` sections but will default to `DATADIR/`**`common`**. (Previously each queue would default to `DATADIR/`**`name`**.)
- `LENGTH`: **20**: Maximal queue size before channel queues block
- `BATCH_LENGTH`: **20**: Batch data before passing to the handler
- `CONN_STR`: **redis://127.0.0.1:6379/0**: Connection string for the redis queue type. Options can be set using query params. Similarly LevelDB options can also be set using: **leveldb://relative/path?option=value** or **leveldb:///absolute/path?option=value**, and will override `DATADIR`
@ -381,11 +381,11 @@ relation to port exhaustion.
- `MAX_ATTEMPTS`: **10**: Maximum number of attempts to create the wrapped queue
- `TIMEOUT`: **GRACEFUL_HAMMER_TIME + 30s**: Timeout the creation of the wrapped queue if it takes longer than this to create.
- Queues by default come with a dynamically scaling worker pool. The following settings configure this:
- `WORKERS`: **1**: Number of initial workers for the queue.
- `WORKERS`: **0** (v1.14 and before: **1**): Number of initial workers for the queue.
- `MAX_WORKERS`: **10**: Maximum number of worker go-routines for the queue.
- `BLOCK_TIMEOUT`: **1s**: If the queue blocks for this time, boost the number of workers - the `BLOCK_TIMEOUT` will then be doubled before boosting again whilst the boost is ongoing.
- `BOOST_TIMEOUT`: **5m**: Boost workers will timeout after this long.
- `BOOST_WORKERS`: **5**: This many workers will be added to the worker pool if there is a boost.
- `BOOST_WORKERS`: **1** (v1.14 and before: **5**): This many workers will be added to the worker pool if there is a boost.
## Admin (`admin`)