GitLab Pages administration (FREE SELF)
- Introduced in GitLab EE 8.3.
- Custom CNAMEs with TLS support were introduced in GitLab EE 8.5.
- GitLab Pages was ported to Community Edition in GitLab 8.17.
- Support for subgroup project's websites was introduced in GitLab 11.8.
GitLab Pages allows for hosting of static sites. It must be configured by an administrator. Separate user documentation is available.
NOTE: This guide is for Omnibus GitLab installations. If you have installed GitLab from source, see GitLab Pages administration for source installations.
Overview
GitLab Pages makes use of the GitLab Pages daemon, a simple HTTP server written in Go that can listen on an external IP address and provide support for custom domains and custom certificates. It supports dynamic certificates through SNI and exposes pages using HTTP2 by default. You are encouraged to read its README to fully understand how it works.
In the case of custom domains (but not
wildcard domains), the Pages daemon needs to listen on
ports 80
and/or 443
. For that reason, there is some flexibility in the way
which you can set it up:
- Run the Pages daemon in the same server as GitLab, listening on a secondary IP.
- Run the Pages daemon in a separate server. In that case, the Pages path must also be present in the server that the Pages daemon is installed, so you must share it through the network.
- Run the Pages daemon in the same server as GitLab, listening on the same IP but on different ports. In that case, you must proxy the traffic with a load balancer. If you choose that route note that you should use TCP load balancing for HTTPS. If you use TLS-termination (HTTPS-load balancing), the pages can't be served with user-provided certificates. For HTTP it's OK to use HTTP or TCP load balancing.
In this document, we proceed assuming the first option. If you are not supporting custom domains a secondary IP is not needed.
Prerequisites
Before proceeding with the Pages configuration, you must:
-
Have a domain for Pages that is not a subdomain of your GitLab instance domain.
GitLab domain Pages domain Does it work? example.com
example.io
{check-circle} Yes example.com
pages.example.com
{dotted-circle} No gitlab.example.com
pages.example.com
{check-circle} Yes -
Configure a wildcard DNS record.
-
(Optional) Have a wildcard certificate for that domain if you decide to serve Pages under HTTPS.
-
(Optional but recommended) Enable Shared runners so that your users don't have to bring their own.
-
(Only for custom domains) Have a secondary IP.
NOTE: If your GitLab instance and the Pages daemon are deployed in a private network or behind a firewall, your GitLab Pages websites are only accessible to devices/users that have access to the private network.
Add the domain to the Public Suffix List
The Public Suffix List is used by browsers to
decide how to treat subdomains. If your GitLab instance allows members of the
public to create GitLab Pages sites, it also allows those users to create
subdomains on the pages domain (example.io
). Adding the domain to the Public
Suffix List prevents browsers from accepting
supercookies,
among other things.
Follow these instructions to submit your
GitLab Pages subdomain. For instance, if your domain is example.io
, you should
request that example.io
is added to the Public Suffix List. GitLab.com
added gitlab.io
in 2016.
DNS configuration
GitLab Pages expect to run on their own virtual host. In your DNS server/provider you need to add a wildcard DNS A record pointing to the host that GitLab runs. For example, an entry would look like this:
*.example.io. 1800 IN A 192.0.2.1
*.example.io. 1800 IN AAAA 2001:db8::1
Where example.io
is the domain GitLab Pages is served from,
192.0.2.1
is the IPv4 address of your GitLab instance, and 2001:db8::1
is the
IPv6 address. If you don't have IPv6, you can omit the AAAA record.
Custom domains
If support for custom domains is needed, the Pages root domain and its subdomains should point to
the secondary IP (which is dedicated for the Pages daemon). <namespace>.<pages root domain>
should
point at Pages directly. Without this, users aren't able to use CNAME
records to point their
custom domains to their GitLab Pages.
For example, an entry could look like this:
example.com 1800 IN A 192.0.2.1
*.example.io. 1800 IN A 192.0.2.2
This example contains the following:
-
example.com
: The GitLab domain. -
example.io
: The domain GitLab Pages is served from. -
192.0.2.1
: The primary IP of your GitLab instance. -
192.0.2.2
: The secondary IP, which is dedicated to GitLab Pages. It must be different than the primary IP.
NOTE: You should not use the GitLab domain to serve user pages. For more information see the security section.
Configuration
Depending on your needs, you can set up GitLab Pages in 4 different ways.
The following examples are listed from the easiest setup to the most advanced one. The absolute minimum requirement is to set up the wildcard DNS since that is needed in all configurations.
Wildcard domains
Requirements:
URL scheme: http://<namespace>.example.io/<project_slug>
This is the minimum setup that you can use Pages with. It is the base for all other setups as described below. NGINX proxies all requests to the daemon. The Pages daemon doesn't listen to the outside world.
-
Set the external URL for GitLab Pages in
/etc/gitlab/gitlab.rb
:pages_external_url 'http://example.io'
Watch the video tutorial for this configuration.
Wildcard domains with TLS support
Requirements:
- Wildcard DNS setup
- Wildcard TLS certificate
URL scheme: https://<namespace>.example.io/<project_slug>
NGINX proxies all requests to the daemon. Pages daemon doesn't listen to the outside world.
-
Place the
example.io
certificate and key inside/etc/gitlab/ssl
. -
In
/etc/gitlab/gitlab.rb
specify the following configuration:pages_external_url 'https://example.io' pages_nginx['redirect_http_to_https'] = true
-
If you haven't named your certificate and key
example.io.crt
andexample.io.key
then you'll need to also add the full paths as shown below:pages_nginx['ssl_certificate'] = "/etc/gitlab/ssl/pages-nginx.crt" pages_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/pages-nginx.key"
-
If you're using Pages Access Control, update the redirect URI in the GitLab Pages System OAuth application to use the HTTPS protocol.
WARNING: Multiple wildcards for one instance is not supported. Only one wildcard per instance can be assigned.
Additional configuration for Docker container
The GitLab Pages daemon doesn't have permissions to bind mounts when it runs
in a Docker container. To overcome this issue, you must change the chroot
behavior:
-
Edit
/etc/gitlab/gitlab.rb
. -
Set the
inplace_chroot
totrue
for GitLab Pages:gitlab_pages['inplace_chroot'] = true
NOTE:
inplace_chroot
option might not work with the other features, such as Pages Access Control.
The GitLab Pages README has more information about caveats and workarounds.
Global settings
Below is a table of all configuration settings known to Pages in Omnibus GitLab,
and what they do. These options can be adjusted in /etc/gitlab/gitlab.rb
,
and take effect after you reconfigure GitLab.
Most of these settings don't need to be configured manually unless you need more granular
control over how the Pages daemon runs and serves content in your environment.
Setting | Description |
---|---|
pages_external_url |
The URL where GitLab Pages is accessible, including protocol (HTTP / HTTPS). If https:// is used, additional configuration is required. See Wildcard domains with TLS support and Custom domains with TLS support for details. |
gitlab_pages[] |
|
access_control |
Whether to enable access control. |
api_secret_key |
Full path to file with secret key used to authenticate with the GitLab API. Auto-generated when left unset. |
artifacts_server |
Enable viewing artifacts in GitLab Pages. |
artifacts_server_timeout |
Timeout (in seconds) for a proxied request to the artifacts server. |
artifacts_server_url |
API URL to proxy artifact requests to. Defaults to GitLab external URL + /api/v4 , for example https://gitlab.com/api/v4 . When running a separate Pages server, this URL must point to the main GitLab server's API. |
auth_redirect_uri |
Callback URL for authenticating with GitLab. Defaults to project's subdomain of pages_external_url + /auth . |
auth_secret |
Secret key for signing authentication requests. Leave blank to pull automatically from GitLab during OAuth registration. |
dir |
Working directory for configuration and secrets files. |
enable |
Enable or disable GitLab Pages on the current system. |
external_http |
Configure Pages to bind to one or more secondary IP addresses, serving HTTP requests. Multiple addresses can be given as an array, along with exact ports, for example ['1.2.3.4', '1.2.3.5:8063'] . Sets value for listen_http . |
external_https |
Configure Pages to bind to one or more secondary IP addresses, serving HTTPS requests. Multiple addresses can be given as an array, along with exact ports, for example ['1.2.3.4', '1.2.3.5:8063'] . Sets value for listen_https . |
gitlab_client_http_timeout |
GitLab API HTTP client connection timeout in seconds (default: 10s). |
gitlab_client_jwt_expiry |
JWT Token expiry time in seconds (default: 30s). |
gitlab_cache_expiry |
The maximum time a domain's configuration is stored in the cache (default: 600s). |
gitlab_cache_refresh |
The interval at which a domain's configuration is set to be due to refresh (default: 60s). |
gitlab_cache_cleanup |
The interval at which expired items are removed from the cache (default: 60s). |
gitlab_retrieval_timeout |
The maximum time to wait for a response from the GitLab API per request (default: 30s). |
gitlab_retrieval_interval |
The interval to wait before retrying to resolve a domain's configuration via the GitLab API (default: 1s). |
gitlab_retrieval_retries |
The maximum number of times to retry to resolve a domain's configuration via the API (default: 3). |
domain_config_source |
Domain configuration source (default: auto ) |
gitlab_id |
The OAuth application public ID. Leave blank to automatically fill when Pages authenticates with GitLab. |
gitlab_secret |
The OAuth application secret. Leave blank to automatically fill when Pages authenticates with GitLab. |
auth_scope |
The OAuth application scope to use for authentication. Must match GitLab Pages OAuth application settings. Leave blank to use api scope by default. |
gitlab_server |
Server to use for authentication when access control is enabled; defaults to GitLab external_url . |
headers |
Specify any additional http headers that should be sent to the client with each response. Multiple headers can be given as an array, header and value as one string, for example ['my-header: myvalue', 'my-other-header: my-other-value']
|
inplace_chroot |
On systems that don't support bind-mounts, this instructs GitLab Pages to chroot into its pages_path directory. Some caveats exist when using in-place chroot ; refer to the GitLab Pages README for more information. |
enable_disk |
Allows the GitLab Pages daemon to serve content from disk. Shall be disabled if shared disk storage isn't available. |
insecure_ciphers |
Use default list of cipher suites, may contain insecure ones like 3DES and RC4. |
internal_gitlab_server |
Internal GitLab server address used exclusively for API requests. Useful if you want to send that traffic over an internal load balancer. Defaults to GitLab external_url . |
listen_proxy |
The addresses to listen on for reverse-proxy requests. Pages binds to these addresses' network sockets and receives incoming requests from them. Sets the value of proxy_pass in $nginx-dir/conf/gitlab-pages.conf . |
log_directory |
Absolute path to a log directory. |
log_format |
The log output format: text or json . |
log_verbose |
Verbose logging, true/false. |
propagate_correlation_id |
Set to true (false by default) to re-use existing Correlation ID from the incoming request header X-Request-ID if present. If a reverse proxy sets this header, the value is propagated in the request chain. |
max_connections |
Limit on the number of concurrent connections to the HTTP, HTTPS or proxy listeners. |
metrics_address |
The address to listen on for metrics requests. |
redirect_http |
Redirect pages from HTTP to HTTPS, true/false. |
sentry_dsn |
The address for sending Sentry crash reporting to. |
sentry_enabled |
Enable reporting and logging with Sentry, true/false. |
sentry_environment |
The environment for Sentry crash reporting. |
status_uri |
The URL path for a status page, for example, /@status . |
tls_max_version |
Specifies the maximum SSL/TLS version ("ssl3", "tls1.0", "tls1.1" or "tls1.2"). |
tls_min_version |
Specifies the minimum SSL/TLS version ("ssl3", "tls1.0", "tls1.1" or "tls1.2"). |
use_http2 |
Enable HTTP2 support. |
gitlab_pages['env'][] |
|
http_proxy |
Configure GitLab Pages to use an HTTP Proxy to mediate traffic between Pages and GitLab. Sets an environment variable http_proxy when starting Pages daemon. |
gitlab_rails[] |
|
pages_domain_verification_cron_worker |
Schedule for verifying custom GitLab Pages domains. |
pages_domain_ssl_renewal_cron_worker |
Schedule for obtaining and renewing SSL certificates through Let's Encrypt for GitLab Pages domains. |
pages_domain_removal_cron_worker |
Schedule for removing unverified custom GitLab Pages domains. |
pages_path |
The directory on disk where pages are stored, defaults to GITLAB-RAILS/shared/pages . |
pages_nginx[] |
|
enable |
Include a virtual host server{} block for Pages inside NGINX. Needed for NGINX to proxy traffic back to the Pages daemon. Set to false if the Pages daemon should directly receive all requests, for example, when using custom domains. |
FF_ENABLE_REDIRECTS |
Feature flag to disable redirects (enabled by default). Read the redirects documentation for more information. |
Advanced configuration
In addition to the wildcard domains, you can also have the option to configure GitLab Pages to work with custom domains. Again, there are two options here: support custom domains with and without TLS certificates. The easiest setup is that without TLS certificates. In either case, you need a secondary IP. If you have IPv6 as well as IPv4 addresses, you can use them both.
Custom domains
Requirements:
- Wildcard DNS setup
- Secondary IP
URL scheme: http://<namespace>.example.io/<project_slug>
and http://custom-domain.com
In that case, the Pages daemon is running, NGINX still proxies requests to the daemon but the daemon is also able to receive requests from the outside world. Custom domains are supported, but no TLS.
-
In
/etc/gitlab/gitlab.rb
specify the following configuration:pages_external_url "http://example.io" nginx['listen_addresses'] = ['192.0.2.1'] # The primary IP of the GitLab instance pages_nginx['enable'] = false gitlab_pages['external_http'] = ['192.0.2.2:80', '[2001:db8::2]:80'] # The secondary IPs for the GitLab Pages daemon
If you don't have IPv6, you can omit the IPv6 address.
Custom domains with TLS support
Requirements:
- Wildcard DNS setup
- Wildcard TLS certificate
- Secondary IP
URL scheme: https://<namespace>.example.io/<project_slug>
and https://custom-domain.com
In that case, the Pages daemon is running, NGINX still proxies requests to the daemon but the daemon is also able to receive requests from the outside world. Custom domains and TLS are supported.
-
Place the
example.io
certificate and key inside/etc/gitlab/ssl
. -
In
/etc/gitlab/gitlab.rb
specify the following configuration:pages_external_url "https://example.io" nginx['listen_addresses'] = ['192.0.2.1'] # The primary IP of the GitLab instance pages_nginx['enable'] = false gitlab_pages['external_http'] = ['192.0.2.2:80', '[2001:db8::2]:80'] # The secondary IPs for the GitLab Pages daemon gitlab_pages['external_https'] = ['192.0.2.2:443', '[2001:db8::2]:443'] # The secondary IPs for the GitLab Pages daemon # Redirect pages from HTTP to HTTPS gitlab_pages['redirect_http'] = true
If you don't have IPv6, you can omit the IPv6 address.
-
If you haven't named your certificate and key
example.io.crt
andexample.io.key
respectively, then you need to also add the full paths as shown below:gitlab_pages['cert'] = "/etc/gitlab/ssl/example.io.crt" gitlab_pages['cert_key'] = "/etc/gitlab/ssl/example.io.key"
-
If you're using Pages Access Control, update the redirect URI in the GitLab Pages System OAuth application to use the HTTPS protocol.
Custom domain verification
To prevent malicious users from hijacking domains that don't belong to them, GitLab supports custom domain verification. When adding a custom domain, users are required to prove they own it by adding a GitLab-controlled verification code to the DNS records for that domain.
If your user base is private or otherwise trusted, you can disable the verification requirement. Go to Admin Area > Settings > Preferences and uncheck Require users to prove ownership of custom domains in the Pages section. This setting is enabled by default.
Let's Encrypt integration
Introduced in GitLab 12.1.
GitLab Pages' Let's Encrypt integration allows users to add Let's Encrypt SSL certificates for GitLab Pages sites served under a custom domain.
To enable it, you must:
- Choose an email address on which you want to receive notifications about expiring domains.
- Go to your instance's Admin Area > Settings > Preferences and expand Pages settings.
- Enter the email address for receiving notifications and accept Let's Encrypt's Terms of Service as shown below.
- Click Save changes.
Access control
Introduced in GitLab 11.5.
GitLab Pages access control can be configured per-project, and allows access to a Pages site to be controlled based on a user's membership to that project.
Access control works by registering the Pages daemon as an OAuth application with GitLab. Whenever a request to access a private Pages site is made by an unauthenticated user, the Pages daemon redirects the user to GitLab. If authentication is successful, the user is redirected back to Pages with a token, which is persisted in a cookie. The cookies are signed with a secret key, so tampering can be detected.
Each request to view a resource in a private site is authenticated by Pages using that token. For each request it receives, it makes a request to the GitLab API to check that the user is authorized to read that site.
Pages access control is disabled by default. To enable it:
-
Enable it in
/etc/gitlab/gitlab.rb
:gitlab_pages['access_control'] = true
-
Users can now configure it in their projects' settings.
NOTE: For this setting to be effective with multi-node setups, it has to be applied to all the App nodes and Sidekiq nodes.
Using Pages with reduced authentication scope
Introduced in GitLab 13.10.
By default, the Pages daemon uses the api
scope to authenticate. You can configure this. For
example, this reduces the scope to read_api
in /etc/gitlab/gitlab.rb
:
gitlab_pages['auth_scope'] = 'read_api'
The scope to use for authentication must match the GitLab Pages OAuth application settings. Users of pre-existing applications must modify the GitLab Pages OAuth application. Follow these steps to do this:
- Go to your instance's Admin Area > Settings > Applications and expand GitLab Pages settings.
- Clear the
api
scope's checkbox and select the desired scope's checkbox (for example,read_api
). - Click Save changes.
Disabling public access to all Pages websites
Introduced in GitLab 12.7.
You can enforce Access Control for all GitLab Pages websites hosted on your GitLab instance. By doing so, only logged-in users have access to them. This setting overrides Access Control set by users in individual projects.
This can be useful to preserve information published with Pages websites to the users of your instance only. To do that:
- Go to your instance's Admin Area > Settings > Preferences and expand Pages settings.
- Check the Disable public access to Pages sites checkbox.
- Click Save changes.
WARNING: For self-managed installations, all public websites remain private until they are redeployed. Resolve this issue by sourcing domain configuration from the GitLab API.
Running behind a proxy
Like the rest of GitLab, Pages can be used in those environments where external internet connectivity is gated by a proxy. To use a proxy for GitLab Pages:
-
Configure in
/etc/gitlab/gitlab.rb
:gitlab_pages['env']['http_proxy'] = 'http://example:8080'
-
Reconfigure GitLab for the changes to take effect.
Using a custom Certificate Authority (CA)
When using certificates issued by a custom CA, Access Control and the online view of HTML job artifacts fails to work if the custom CA is not recognized.
This usually results in this error:
Post /oauth/token: x509: certificate signed by unknown authority
.
For installation from source, this can be fixed by installing the custom Certificate Authority (CA) in the system certificate store.
For Omnibus, this is fixed by installing a custom CA in Omnibus GitLab.
Zip serving and cache configuration
Introduced in GitLab 13.7.
WARNING: These are advanced settings. The recommended default values are set inside GitLab Pages. You should change these settings only if absolutely necessary. Use extreme caution.
GitLab Pages can serve content from zip archives through object storage (an issue exists for supporting disk storage as well). It uses an in-memory cache to increase the performance when serving content from a zip archive. You can modify the cache behavior by changing the following configuration flags.
Setting | Description |
---|---|
zip_cache_expiration |
The cache expiration interval of zip archives. Must be greater than zero to avoid serving stale content. Default is 60s. |
zip_cache_cleanup |
The interval at which archives are cleaned from memory if they have already expired. Default is 30s. |
zip_cache_refresh |
The time interval in which an archive is extended in memory if accessed before zip_cache_expiration . This works together with zip_cache_expiration to determine if an archive is extended in memory. See the example below for important details. Default is 30s. |
zip_open_timeout |
The maximum time allowed to open a zip archive. Increase this time for big archives or slow network connections, as doing so may affect the latency of serving Pages. Default is 30s. |
Zip cache refresh example
Archives are refreshed in the cache (extending the time they are held in memory) if they're accessed
before zip_cache_expiration
, and the time left before expiring is less than or equal to
zip_cache_refresh
. For example, if archive.zip
is accessed at time 0s, it expires in 60s (the
default for zip_cache_expiration
). In the example below, if the archive is opened again after 15s
it is not refreshed because the time left for expiry (45s) is greater than zip_cache_refresh
(default 30s). However, if the archive is accessed again after 45s (from the first time it was
opened) it's refreshed. This extends the time the archive remains in memory from
45s + zip_cache_expiration (60s)
, for a total of 105s.
After an archive reaches zip_cache_expiration
, it's marked as expired and removed on the next
zip_cache_cleanup
interval.
Activate verbose logging for daemon
Verbose logging was introduced in Omnibus GitLab 11.1.
Follow the steps below to configure verbose logging of GitLab Pages daemon.
-
By default the daemon only logs with
INFO
level. If you wish to make it log events with levelDEBUG
you must configure this in/etc/gitlab/gitlab.rb
:gitlab_pages['log_verbose'] = true
Propagating the correlation ID
Introduced in GitLab 13.10.
Setting the propagate_correlation_id
to true allows installations behind a reverse proxy to generate
and set a correlation ID to requests sent to GitLab Pages. When a reverse proxy sets the header value X-Request-ID
,
the value propagates in the request chain.
Users can find the correlation ID in the logs.
To enable the propagation of the correlation ID:
-
Set the parameter to true in
/etc/gitlab/gitlab.rb
:gitlab_pages['propagate_correlation_id'] = true
Change storage path
Follow the steps below to change the default path where GitLab Pages' contents are stored.
-
Pages are stored by default in
/var/opt/gitlab/gitlab-rails/shared/pages
. If you wish to store them in another location you must set it up in/etc/gitlab/gitlab.rb
:gitlab_rails['pages_path'] = "/mnt/storage/pages"
Alternatively, if you have existing Pages deployed you can follow the below steps to do a no downtime transfer to a new storage location.
-
Pause Pages deployments by setting the following in
/etc/gitlab/gitlab.rb
:sidekiq['queue_selector'] = true sidekiq['queue_groups'] = [ "feature_category!=pages" ]
-
rsync
contents from the current storage location to the new storage location:sudo rsync -avzh --progress /var/opt/gitlab/gitlab-rails/shared/pages/ /mnt/storage/pages
-
Set the new storage location in
/etc/gitlab/gitlab.rb
:gitlab_rails['pages_path'] = "/mnt/storage/pages"
-
Verify Pages are still being served up as expected.
-
Resume Pages deployments by removing from
/etc/gitlab/gitlab.rb
thesidekiq
setting set above. -
Trigger a new Pages deployment and verify it's working as expected.
-
Remove the old Pages storage location:
sudo rm -rf /var/opt/gitlab/gitlab-rails/shared/pages
-
Verify Pages are still being served up as expected.
Configure listener for reverse proxy requests
Introduced in Omnibus GitLab 11.1.
Follow the steps below to configure the proxy listener of GitLab Pages.
-
By default the listener is configured to listen for requests on
localhost:8090
.If you wish to disable it you must configure this in
/etc/gitlab/gitlab.rb
:gitlab_pages['listen_proxy'] = nil
If you wish to make it listen on a different port you must configure this also in
/etc/gitlab/gitlab.rb
:gitlab_pages['listen_proxy'] = "localhost:10080"
Set maximum pages size
You can configure the maximum size of the unpacked archive per project in Admin Area > Settings > Preferences > Pages, in Maximum size of pages (MB). The default is 100MB.
Override maximum pages size per project or group (PREMIUM SELF)
Introduced in GitLab 12.7.
NOTE: Only GitLab admin users are able to view and override the Maximum size of Pages setting.
To override the global maximum pages size for a specific project:
- Go to your project's Settings > Pages page.
- Edit the Maximum size of pages.
- Click Save changes.
To override the global maximum pages size for a specific group:
- Go to your group's Settings > General page and expand Pages.
- Edit the Maximum size of pages.
- Click Save changes.
Running GitLab Pages on a separate server
You can run the GitLab Pages daemon on a separate server to decrease the load on your main application server.
To configure GitLab Pages on a separate server:
WARNING:
The following procedure includes steps to back up and edit the
gitlab-secrets.json
file. This file contains secrets that control
database encryption. Proceed with caution.
-
Create a backup of the secrets file on the GitLab server:
cp /etc/gitlab/gitlab-secrets.json /etc/gitlab/gitlab-secrets.json.bak
-
On the GitLab server, to enable Pages, add the following to
/etc/gitlab/gitlab.rb
:pages_external_url "http://<pages_server_URL>"
-
Optionally, to enable access control, add the following to
/etc/gitlab/gitlab.rb
:gitlab_pages['access_control'] = true
-
Reconfigure the GitLab server for the changes to take effect. The
gitlab-secrets.json
file is now updated with the new configuration. -
Set up a new server. This becomes the Pages server.
-
Create an NFS share on the Pages server and configure this share to allow access from your main GitLab server. Note that the example there is more general and shares several sub-directories from
/home
to several/nfs/home
mount points. For our Pages-specific example here, we instead share only the default GitLab Pages folder/var/opt/gitlab/gitlab-rails/shared/pages
from the Pages server and we mount it to/mnt/pages
on the GitLab server. Therefore, omit "Step 4" there. -
On the Pages server, install Omnibus GitLab and modify
/etc/gitlab/gitlab.rb
to include:roles ['pages_role'] pages_external_url "http://<pages_server_URL>" gitlab_pages['gitlab_server'] = 'http://<gitlab_server_IP_or_URL>'
-
Create a backup of the secrets file on the Pages server:
cp /etc/gitlab/gitlab-secrets.json /etc/gitlab/gitlab-secrets.json.bak
-
Copy the
/etc/gitlab/gitlab-secrets.json
file from the GitLab server to the Pages server, for example via the NFS share.# On the GitLab server cp /etc/gitlab/gitlab-secrets.json /mnt/pages/gitlab-secrets.json # On the Pages server mv /var/opt/gitlab/gitlab-rails/shared/pages/gitlab-secrets.json /etc/gitlab/gitlab-secrets.json
-
Reconfigure GitLab for the changes to take effect.
-
On the GitLab server, make the following changes to
/etc/gitlab/gitlab.rb
:pages_external_url "http://<pages_server_URL>" gitlab_pages['enable'] = false pages_nginx['enable'] = false gitlab_rails['pages_path'] = "/mnt/pages"
-
Reconfigure GitLab for the changes to take effect.
It's possible to run GitLab Pages on multiple servers if you wish to distribute the load. You can do this through standard load balancing practices such as configuring your DNS server to return multiple IPs for your Pages server, configuring a load balancer to work at the IP level, and so on. If you wish to set up GitLab Pages on multiple servers, perform the above procedure for each Pages server.
Domain source configuration
Introduced in GitLab 13.3.
GitLab Pages can use different sources to get domain configuration.
The default value for Omnibus installations is nil
.
gitlab_pages['domain_config_source'] = nil
If left unchanged, GitLab Pages tries to use any available source (either gitlab
or disk
). The
preferred source is gitlab
, which uses API-based configuration.
On large GitLab instances, using the API-based configuration significantly improves the pages daemon startup time, as there is no need to load all custom domains configuration into memory.
For more details see this blog post.
domain_config_source
Deprecated WARNING:
The flag gitlab_pages['domain_config_source']
is deprecated for use in GitLab 13.9,
and is planned for removal in GitLab 14.0.
GitLab 13.0 introduced the special flag domain_config_source
to support manual opt-in to
API-based configuration.
GitLab 13.7 introduced the auto
value
to support a smoother transition to API-based configuration.
Starting with GitLab 14.0, GitLab Pages only supports API-based configuration, and
disk source configuration is removed.
Therefore, GitLab 14.0 also removes domain_config_source
.
GitLab Pages fails to start if it can't connect to the GitLab API. For other common issues, see the troubleshooting section or report an issue.
GitLab API-based configuration
WARNING:
The flag gitlab_pages['domain_config_source']
is deprecated for use in GitLab 13.9,
and is planned for removal in GitLab 14.0. In GitLab 14.0 and later, GitLab Pages attempts to
connect to the API automatically, without requiring the manual configuration steps shown here. Pages
fails to start if this automatic connection fails.
GitLab Pages can use an API-based configuration. This replaces disk source configuration, which was used prior to GitLab 13.0. Follow these steps to enable it:
-
Add the following to your
/etc/gitlab/gitlab.rb
file:gitlab_pages['domain_config_source'] = "gitlab"
-
Reconfigure GitLab for the changes to take effect.
If you encounter an issue, you can disable it by choosing disk
:
gitlab_pages['domain_config_source'] = "disk"
For other common issues, see the troubleshooting section or report an issue.
GitLab API cache configuration
Introduced in GitLab 13.10.
API-based configuration uses a caching mechanism to improve performance and reliability of serving Pages. The cache behavior can be modified by changing the cache settings, however, the recommended values are set for you and should only be modified if needed. Incorrect configuration of these values may result in intermittent or persistent errors, or the Pages Daemon serving old content.
NOTE: Expiry, interval and timeout flags use Golang's duration formatting. A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as "300ms", "1.5h" or "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".
Examples:
-
Increasing
gitlab_cache_expiry
allows items to exist in the cache longer. This setting might be useful if the communication between GitLab Pages and GitLab Rails is not stable. -
Increasing
gitlab_cache_refresh
reduces the frequency at which GitLab Pages requests a domain's configuration from GitLab Rails. This setting might be useful GitLab Pages generates too many requests to GitLab API and content does not change frequently. -
Decreasing
gitlab_cache_cleanup
removes expired items from the cache more frequently, reducing the memory usage of your Pages node. -
Decreasing
gitlab_retrieval_timeout
allows you to stop the request to GitLab Rails more quickly. Increasing it allows more time to receive a response from the API, useful in slow networking environments. -
Decreasing
gitlab_retrieval_interval
makes requests to the API more frequently, only when there is an error response from the API, for example a connection timeout. -
Decreasing
gitlab_retrieval_retries
reduces the number of times a domain's configuration is tried to be resolved automatically before reporting an error.
Using object storage
Introduced in GitLab 13.6.
Read more about using object storage with GitLab.
Object storage settings
The following settings are:
- Nested under
pages:
and thenobject_store:
on source installations. - Prefixed by
pages_object_store_
on Omnibus GitLab installations.
Setting | Description | Default |
---|---|---|
enabled |
Whether object storage is enabled. | false |
remote_directory |
The name of the bucket where Pages site content is stored. | |
connection |
Various connection options described below. |
NOTE: If you want to stop using and disconnect the NFS server, you need to explicitly disable local storage, and it's only possible after upgrading to GitLab 13.11.
S3-compatible connection settings
See the available connection settings for different providers.
In Omnibus installations:
-
Add the following lines to
/etc/gitlab/gitlab.rb
and replace the values with the ones you want:gitlab_rails['pages_object_store_enabled'] = true gitlab_rails['pages_object_store_remote_directory'] = "pages" gitlab_rails['pages_object_store_connection'] = { 'provider' => 'AWS', 'region' => 'eu-central-1', 'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', 'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY' }
If you use AWS IAM profiles, be sure to omit the AWS access key and secret access key/value pairs:
gitlab_rails['pages_object_store_connection'] = { 'provider' => 'AWS', 'region' => 'eu-central-1', 'use_iam_profile' => true }
-
Save the file and reconfigure GitLab for the changes to take effect.
In installations from source:
-
Edit
/home/git/gitlab/config/gitlab.yml
and add or amend the following lines:pages: object_store: enabled: true remote_directory: "pages" # The bucket name connection: provider: AWS # Only AWS supported at the moment aws_access_key_id: AWS_ACCESS_KEY_ID aws_secret_access_key: AWS_SECRET_ACCESS_KEY region: eu-central-1
-
Save the file and restart GitLab for the changes to take effect.
ZIP storage
In GitLab 14.0 the underlaying storage format of GitLab Pages is changing from files stored directly in disk to a single ZIP archive per project.
These ZIP archives can be stored either locally on disk storage or on the object storage if it is configured.
Starting from GitLab 13.5 ZIP archives are stored every time pages site is updated.
Migrate legacy storage to ZIP storage
Introduced in GitLab 13.11.
GitLab tries to automatically migrate the old storage format to the new ZIP-based one when you upgrade to GitLab 13.11 or further. However, some projects may fail to be migrated for different reasons. To verify that all projects have been migrated successfully, you can manually run the migration:
gitlab-rake gitlab:pages:migrate_legacy_storage
It's safe to interrupt this task and run it multiple times.
There are two most common problems this task can report:
-
Missing public directory
error:E, [2021-04-09T13:11:52.534768 #911919] ERROR -- : project_id: 1 /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test failed to be migrated in 0.07 seconds: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test
In this case, you should verify that these projects don't have pages deployed, and re-run the migration with an additional flag to mark those projects as not deployed with GitLab Pages:
sudo PAGES_MIGRATION_MARK_PROJECTS_AS_NOT_DEPLOYED=true gitlab-rake gitlab:pages:migrate_legacy_storage
-
File
is invalid
error:E, [2021-04-09T14:43:05.821767 #923322] ERROR -- : project_id: 1 /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test failed to be migrated: /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test/public/link is invalid, input_dir: /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test
This error indicates invalid files on disk storage, most commonly symlinks leading outside of the
public
directory. You can manually remove these files, or just ignore them during migration:sudo PAGES_MIGRATION_IGNORE_INVALID_ENTRIES=true gitlab-rake gitlab:pages:migrate_legacy_storage
Rolling back ZIP migration
If you find that migrated data is invalid, you can remove all migrated data by running:
sudo gitlab-rake gitlab:pages:clean_migrated_zip_storage
This does not remove any data from the legacy disk storage and the GitLab Pages daemon automatically falls back to using that.
Migrate Pages deployments to object storage
Introduced in GitLab 13.11.
Existing Pages deployments objects (which store ZIP archives) can similarly be migrated to object storage, if you've been having them stored locally.
Migrate your existing Pages deployments from local storage to object storage:
sudo gitlab-rake gitlab:pages:deployments:migrate_to_object_storage
Rolling Pages deployments back to local storage
After the migration to object storage is performed, you can choose to revert your Pages deployments back to local storage:
sudo gitlab-rake gitlab:pages:deployments:migrate_to_local
Disable Pages local storage
Introduced in GitLab 13.11.
If you use object storage, disable local storage:
-
Edit
/etc/gitlab/gitlab.rb
:gitlab_rails['pages_local_store_enabled'] = false
-
Reconfigure GitLab for the changes to take effect.
Starting from GitLab 13.12, this setting also disables the legacy storage, so if you were using NFS to serve Pages, you can completely disconnect from it.
Migrate GitLab Pages to 14.0
In GitLab 14.0 a number of breaking changes are introduced which may require some user intervention. The steps below describe the best way to migrate without causing any downtime for your GitLab instance.
If you run GitLab on a single server, then most likely you will not notice any problem after upgrading to GitLab 14.0, but it may be safer to follow the steps anyway. If you run GitLab on a single server, then most likely the upgrade process to 14.0 will go smoothly for you. Regardless, we recommend everyone follow the migration steps to ensure a successful upgrade. If at any point you run into issues, consult the troubleshooting section.
To migrate GitLab Pages to GitLab 14.0:
- If your current GitLab version is lower than 13.12, then you first need to upgrade to 13.12. Upgrading directly to 14.0 may cause downtime for some web-sites hosted on GitLab Pages until you finish the following steps.
- Enable the API-based configuration, which is the default starting from GitLab 14.0. Skip this step if you're already running GitLab 14.0 or above.
- If you want to store your pages content in the object storage, make sure to configure it. If you want to store the pages content locally or continue using an NFS server, skip this step.
- Migrate legacy storage to ZIP storage.
Backup
GitLab Pages are part of the regular backup, so there is no separate backup to configure.
Security
You should strongly consider running GitLab Pages under a different hostname than GitLab to prevent XSS attacks.
Troubleshooting
open /etc/ssl/ca-bundle.pem: permission denied
GitLab Pages runs inside a chroot
jail, usually in a uniquely numbered directory like
/tmp/gitlab-pages-*
.
Within the jail, a bundle of trusted certificates is
provided at /etc/ssl/ca-bundle.pem
. It's
copied there
from /opt/gitlab/embedded/ssl/certs/cacert.pem
as part of starting up Pages.
If the permissions on the source file are incorrect (they should be 0644
), then
the file inside the chroot
jail is also wrong.
Pages logs errors in /var/log/gitlab/gitlab-pages/current
like:
x509: failed to load system roots and no roots provided
open /etc/ssl/ca-bundle.pem: permission denied
The use of a chroot
jail makes this error misleading, as it is not
referring to /etc/ssl
on the root file system.
The fix is to correct the source file permissions and restart Pages:
sudo chmod 644 /opt/gitlab/embedded/ssl/certs/cacert.pem
sudo gitlab-ctl restart gitlab-pages
dial tcp: lookup gitlab.example.com
and x509: certificate signed by unknown authority
When setting both inplace_chroot
and access_control
to true
, you might encounter errors like:
dial tcp: lookup gitlab.example.com on [::1]:53: dial udp [::1]:53: connect: cannot assign requested address
Or:
open /opt/gitlab/embedded/ssl/certs/cacert.pem: no such file or directory
x509: certificate signed by unknown authority
The reason for those errors is that the files resolv.conf
and ca-bundle.pem
are missing inside the chroot
.
The fix is to copy the host's /etc/resolv.conf
and the GitLab certificate bundle inside the chroot
:
sudo mkdir -p /var/opt/gitlab/gitlab-rails/shared/pages/etc/ssl
sudo mkdir -p /var/opt/gitlab/gitlab-rails/shared/pages/opt/gitlab/embedded/ssl/certs/
sudo cp /etc/resolv.conf /var/opt/gitlab/gitlab-rails/shared/pages/etc
sudo cp /opt/gitlab/embedded/ssl/certs/cacert.pem /var/opt/gitlab/gitlab-rails/shared/pages/opt/gitlab/embedded/ssl/certs/
sudo cp /opt/gitlab/embedded/ssl/certs/cacert.pem /var/opt/gitlab/gitlab-rails/shared/pages/etc/ssl/ca-bundle.pem
502 error when connecting to GitLab Pages proxy when server does not listen over IPv6
In some cases, NGINX might default to using IPv6 to connect to the GitLab Pages
service even when the server does not listen over IPv6. You can identify when
this is happening if you see something similar to the log entry below in the
gitlab_pages_error.log
:
2020/02/24 16:32:05 [error] 112654#0: *4982804 connect() failed (111: Connection refused) while connecting to upstream, client: 123.123.123.123, server: ~^(?<group>.*)\.pages\.example\.com$, request: "GET /-/group/project/-/jobs/1234/artifacts/artifact.txt HTTP/1.1", upstream: "http://[::1]:8090//-/group/project/-/jobs/1234/artifacts/artifact.txt", host: "group.example.com"
To resolve this, set an explicit IP and port for the GitLab Pages listen_proxy
setting
to define the explicit address that the GitLab Pages daemon should listen on:
gitlab_pages['listen_proxy'] = '127.0.0.1:8090'
Intermittent 502 errors or after a few days
If you run Pages on a system that uses systemd
and
tmpfiles.d
,
you may encounter intermittent 502 errors trying to serve Pages with an error similar to:
dial tcp: lookup gitlab.example.com on [::1]:53: dial udp [::1]:53: connect: no route to host"
GitLab Pages creates a bind mount
inside /tmp/gitlab-pages-*
that includes files like /etc/hosts
.
However, systemd
may clean the /tmp/
directory on a regular basis so the DNS
configuration may be lost.
To stop systemd
from cleaning the Pages related content:
-
Tell
tmpfiles.d
to not remove the Pages/tmp
directory:echo 'x /tmp/gitlab-pages-*' >> /etc/tmpfiles.d/gitlab-pages-jail.conf
-
Restart GitLab Pages:
sudo gitlab-ctl restart gitlab-pages
404 error after transferring the project to a different group or user, or changing project path
If you encounter a 404 Not Found
error a Pages site after transferring a project to
another group or user, or changing project path, you must trigger a domain configuration
update for Pages. To do so, write something in the .update
file. The Pages daemon
monitors for changes to this file, and reloads the configuration when changes occur.
Use this example to fix a 404 Not Found
error after transferring a project or changing
a project path with Pages:
date > /var/opt/gitlab/gitlab-rails/shared/pages/.update
If you've customized the Pages storage path, adjust the command above to use your custom path.
404 error after promoting a Geo secondary to a primary node
These are due to the Pages files not being among the supported data types.
It is possible to copy the subfolders and files in the Pages path
to the new primary node to resolve this.
For example, you can adapt the rsync
strategy from the
moving repositories documenation.
Alternatively, run the CI pipelines of those projects that contain a pages
job again.
Failed to connect to the internal GitLab API
If you have enabled API-based configuration and see the following error:
ERRO[0010] Failed to connect to the internal GitLab API after 0.50s error="failed to connect to internal Pages API: HTTP status: 401"
If you are Running GitLab Pages on a separate server
you must copy the /etc/gitlab/gitlab-secrets.json
file
from the GitLab server to the Pages server after upgrading to GitLab 13.3,
as described in that section.
Other reasons may include network connectivity issues between your GitLab server and your Pages server such as firewall configurations or closed ports. For example, if there is a connection timeout:
error="failed to connect to internal Pages API: Get \"https://gitlab.example.com:3000/api/v4/internal/pages/status\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Pages cannot communicate with an instance of the GitLab API
WARNING:
The flag gitlab_pages['domain_config_source']
is deprecated
for use in GitLab 13.9,
and is planned for removal in GitLab 14.0.
If you use the default value for domain_config_source=auto
and run multiple instances of GitLab
Pages, you may see intermittent 502 error responses while serving Pages content. You may also see
the following warning in the Pages logs:
WARN[0010] Pages cannot communicate with an instance of the GitLab API. Please sync your gitlab-secrets.json file https://gitlab.com/gitlab-org/gitlab-pages/-/issues/535#workaround. error="pages endpoint unauthorized"
This can happen if your gitlab-secrets.json
file is out of date between GitLab Rails and GitLab
Pages. Follow steps 8-10 of Running GitLab Pages on a separate server,
in all of your GitLab Pages instances.
securecookie: failed to generate random iv
and Failed to save the session
500 error with This problem most likely results from an out-dated operating system.
The Pages daemon uses the securecookie
library to get random strings via crypto/rand
in Go.
This requires the getrandom
system call or /dev/urandom
to be available on the host OS.
Upgrading to an officially supported operating system is recommended.
The requested scope is invalid, malformed, or unknown
This problem comes from the permissions of the GitLab Pages OAuth application. To fix it, go to
Admin > Applications > GitLab Pages and edit the application. Under Scopes, ensure that the
api
scope is selected and save your changes.
When running a separate Pages server,
this setting needs to be configured on the main GitLab server.
Workaround in case no wildcard DNS entry can be set
If the wildcard DNS prerequisite can't be met, you can still use GitLab Pages in a limited fashion:
-
Move
all projects you need to use Pages with into a single group namespace, for example
pages
. - Configure a DNS entry without the
*.
-wildcard, for examplepages.example.io
. - Configure
pages_external_url http://example.io/
in yourgitlab.rb
file. Omit the group namespace here, because it automatically is prepended by GitLab.
Pages daemon fails with permission denied errors
If /tmp
is mounted with noexec
, the Pages daemon fails to start with an error like:
{"error":"fork/exec /gitlab-pages: permission denied","level":"fatal","msg":"could not create pages daemon","time":"2021-02-02T21:54:34Z"}
In this case, change TMPDIR
to a location that is not mounted with noexec
. Add the following to
/etc/gitlab/gitlab.rb
:
gitlab_pages['env'] = {'TMPDIR' => '<new_tmp_path>'}
Once added, reconfigure with sudo gitlab-ctl reconfigure
and restart GitLab with
sudo gitlab-ctl restart
.
The redirect URI included is not valid.
when using Pages Access Control
Verify that the Callback URL/Redirect URI in the GitLab Pages System OAuth application
is using the protocol (HTTP or HTTPS) that pages_external_url
is configured to use.
cannot serve from disk
500 error If you get a 500 response from Pages and encounter an error similar to:
ERRO[0145] cannot serve from disk error="gitlab: disk access is disabled via enable-disk=false" project_id=27 source_path="file:///shared/pages/@hashed/67/06/670671cd97404156226e507973f2ab8330d3022ca96e0c93bdbdb320c41adcaf/pages_deployments/14/artifacts.zip" source_type=zip
It means that GitLab Rails is telling GitLab Pages to serve content from a location on disk, however, GitLab Pages was configured to disable disk access.
To enable disk access:
-
Enable disk access for GitLab Pages in
/etc/gitlab/gitlab.rb
:gitlab_pages['enable_disk'] = true