Private connectivity with GCP Private Service Connect

ECH

You can use GCP Private Service Connect to establish a secure connection for your Elastic Cloud deployments to communicate with other GCP services. GCP routes the Private Link traffic within the GCP data center and never exposes it to the public internet.

GCP Private Service Connect connects your Virtual Private Cloud (VPC) to the GCP-hosted services that you use, treating them as if they were in your VPC. You can create and use VPC endpoints to securely access GCP-hosted services.

You can also optionally filter traffic to your deployments by creating virtual private connection (VPC) filters as part of your private connection policy in Elastic Cloud. This limits traffic to your deployment to the VPC specified in the policy, as well as any other policies applied to the deployment.

Private Link is a connection between a Private Service Connect Endpoint and a Service Attachment. Learn more about using Private Service Connect on Google Cloud.

To learn how private connection policies impact your deployment, refer to Network security policies in Elastic Cloud.

• Private connectivity with Private Service Connect is supported only in Google Cloud regions.

• Private Service Connect connections are regional. As a result, your Private Service Connect endpoint needs to be created in the same region as your deployment. The endpoint can be accessed from any region after you enable its Global Access feature.

Service Attachments are set up by Elastic in all supported GCP regions under the following URIs:

The process of setting up the Private link connection to your deployments is split between Google Cloud and the Elastic Cloud UI. These are the high-level steps:

After you create your private connection policy, you can edit, disassociate, or delete it.

1. Create a Private Service Connect endpoint in your VPC using the Service Attachment URI for your region.

   Follow the Google Cloud instructions for details on creating a Private Service Connect endpoint to access Private Service Connect services.

   Use the Service Attachment URI for your region. Select the Published service option and enter the selected Service Attachment URI as the Target service. For example, for the region asia-southeast1 the Service Attachment URI is projects/cloud-production-168820/regions/asia-southeast1/serviceAttachments/proxy-psc-production-asia-southeast1-v1-attachment

   Note

   you need to reserve a static internal IP address in your VPC. The address is used by Private Service Connect endpoint.

   Note down the PSC Connection ID, e.g. 18446744072646845332.

2. Create a DNS record.

   1. Create a DNS Zone of type Private.

      Refer to the Private zone DNS name column in the Private Service Connect URIs table for the name of the zone. For example, in asia-southeast1, use psc.asia-southeast1.gcp.elastic-cloud.com as the zone domain name. Make sure the zone is associated with your VPC.

   2. Create a DNS record set with an A record pointing to the Private Service Connect endpoint IP. Use * as the DNS name, A as the Resource record type, and put the Private Service Connect endpoint IP address as the record value.

      Follow the Google Cloud instructions for details on creating an A record which points to your Private Service Connect endpoint IP address.

After you create your Private Service Connect endpoint and DNS entries, verify that you are able to reach your cluster over Private Link.

To test the connection:

1. If needed, find the endpoint of an application in your deployment:

   1. Log in to the Elastic Cloud Console.

   2. Under Hosted deployments, find your deployment.

   Tip

   If you have many deployments, you can instead go to the Hosted deployments (Elastic Cloud Hosted) page. On that page, you can narrow your deployments by name, ID, or choose from several other filters.

   3. Select Manage.
   4. In the deployment overview, under Applications, find the application that you want to test.
   5. Click Copy endpoint. The value looks something like the following:

   https://my-deployment-d53192.es.us-central1.gcp.cloud.es.io
   

   In this endpoint, my-deployment-d53192 is an alias, and es is the product you want to access within your deployment.

2. Access your cluster over Private Link:

   • If you have a custom endpoint alias configured, you can use the custom endpoint URL to connect.
   • Test the setup using the following cURL command. Pass the username and password for a user that has access to the cluster. Make sure to replace the URL with your deployment's endpoint information and the private hosted zone domain name that you registered.

   Request

   $ curl -v https://my-deployment-d53192.es.psc.asia-southeast1.gcp.elastic-cloud.com:9243 -u {username}:{password}
   

   Response

   ..
   *   Trying 192.168.100.2...
   ..
       < HTTP/1.1 200 OK
   ..
   {
       "name" : "instance-0000000009",
       "cluster_name" : "fb7e805e5cfb4931bdccc4f3cb591f5f",
       "cluster_uuid" : "2cTHeCQYS2a0iH7YnQHrIQ",
       "version" : { ... },
       "tagline" : "You Know, for Search"
   }
   

Check the IP address. it should be the same as the IP address assigned to your Private Service Connect endpoint.

The connection is established, and a valid certificate is presented to the client. Elastic responds, in the case of the Elasticsearch endpoint, with basic information about the cluster.

After you test your Private Link connection, you can create a private connection policy in Elastic Cloud.

Private connection policies are optional for GCP Private Service Connect. After the Private Service Connect endpoint and DNS record are created, private connectivity is established.

Creating a private connection policy and associating it with your deployments allows you to do the following:

• Record that you've established private connectivity between GCP and Elastic in the applicable region.
• Filter traffic to your deployment using VPC filters.

Follow these high-level steps to add a private connection policy that can be associated with your deployments.

1. Optional: Find your Private Service Connect connection ID.
2. Create policies using the Private Service Connect endpoint connection ID.
3. Associate the Private Service Connect endpoint with your deployment.

The PSC connection ID is only required if you want to filter traffic to your deployment using VPC filters.

1. Go to your Private Service Connect endpoint in the Google Cloud console.
2. Copy the value of PSC Connection ID.

Create a new private connection policy.

1. Log in to the Elastic Cloud Console.
2. From any deployment or project on the home page, select Manage.
3. From the left navigation menu, select Access and security > Network security.

4. Select Private connection.

5. Select the resource type that the private connection will be applied to. Currently, only hosted deployments are supported.

6. Select the cloud provider and region for the private connection.

   Tip

   Private connection policies are bound to a single region, and can be assigned only to deployments in the same region. If you want to associate a policy with resources in multiple regions, then you have to create the same policy in all the regions you want to apply it to.

7. Under Connectivity, select Privatelink.

8. Optional: Under VPC filter, enter your Private Service Connect endpoint connection ID. You should only specify a Private Service Connect endpoint connection ID if you want to filter traffic to your deployment.

   If you don't specify a VPC filter, then the private connection policy acts only as a record that you've established private connectivity between AWS and Elastic in the applicable region.

   Tip

   You can apply multiple policies to a single deployment. The policies can be of different types. In case of multiple policies, traffic can match any associated policy to be forwarded to the resource. If none of the policies match, the request is rejected with 403 Forbidden.

   Learn more about how network security policies affect your deployment.

9. Optional: Under Apply to resources, associate the new private connection policy with one or more deployments. If you specified a VPC filter, then after you associate the filter with a deployment, it starts filtering traffic.

10. To automatically attach this private connection policy to new deployments, select Apply by default.

11. Click Create.

12. (Optional) You can claim your Private Service Connect endpoint connection ID, so that no other organization is able to use it in a private connection policy.

The next step is to associate the policy with your deployment.

You can associate a private connection policy with your deployment from the policy's settings, or from your deployment's settings.

If the policy contains a VPC filter, then after you associate the policy with a deployment, it starts filtering traffic.

If the policy doesn't contain a VPC filter, then the association can serve as a reminder that a Private Service Connect endpoint exists for the deployment's region.

1. Find your deployment on the home page or on the Hosted deployments page, then select Manage to access its settings menus.

   On the Hosted deployments page, you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.

2. On the Security page, under Network security, select Apply policies > Private connection.

3. Choose the policy you want to apply and select Apply.

1. Log in to the Elastic Cloud Console.
2. From any deployment or project on the home page, select Manage.
3. From the left navigation menu, select Access and security > Network security.

5. Find the policy you want to edit.
6. Under Apply to resources, associate the policy with one or more deployments.
7. Click Update to save your changes.

For traffic to connect with the deployment over Private Service Connect, the client making the request needs to be located within the VPC where you’ve created the Private Service Connect endpoint. You can also set up network traffic to flow through the originating VPC from somewhere else, such as another VPC or a VPN from your corporate network. This assumes that the Private Service Connect endpoint and the DNS record are also available within that context. Check your cloud service provider documentation for setup instructions.

Use the following URL structure. This URL is built from endpoint information retrieved from your Elastic deployment and the private hosted zone domain name that you registered.

https://{{alias}}.{{product}}.{{private_hosted_zone_domain_name}}

For example:

https://my-deployment-d53192.es.psc.asia-southeast1.gcp.elastic-cloud.com

To access the deployment:

1. If needed, find the endpoint of an application in your deployment:

   1. Log in to the Elastic Cloud Console.

   2. Under Hosted deployments, find your deployment.

   Tip

   If you have many deployments, you can instead go to the Hosted deployments (Elastic Cloud Hosted) page. On that page, you can narrow your deployments by name, ID, or choose from several other filters.

   3. Select Manage.
   4. In the deployment overview, under Applications, find the application that you want to test.
   5. Click Copy endpoint. The value looks something like the following:

   https://my-deployment-d53192.es.us-central1.gcp.cloud.es.io
   

   In this endpoint, my-deployment-d53192 is an alias, and es is the product you want to access within your deployment.

2. Send a request:

   Request

   $ curl -v https://my-deployment-d53192.es.psc.asia-southeast1.gcp.elastic-cloud.com:9243 -u {username}:{password}
   

   Response

   < HTTP/1.1 200 OK
   ..
   {
       "name" : "instance-0000000009",
       "cluster_name" : "fb7e805e5cfb4931bdccc4f3cb591f5f",
       "cluster_uuid" : "2cTHeCQYS2a0iH7YnQHrIQ",
       "version" : { ... },
       "tagline" : "You Know, for Search"
   }
   

If you are using Private Service Connect together with Fleet, and enrolling the Elastic Agent with a private connection URL, you need to configure Fleet Server to use and propagate the Private Service Connect URL by updating the Fleet Server hosts field in the Fleet settings section of Kibana. Otherwise, Elastic Agent will reset to use a default address instead of the Private Service Connect URL.

The URL needs to follow this pattern:

https://{{fleet_component_ID_or_deployment_alias}}.fleet.{{private_hosted_zone_domain_name}}:443`

Similarly, the Elasticsearch host needs to be updated to propagate the private connection URL. The Elasticsearch URL needs to follow this pattern:

https://{{elasticsearch_cluster_ID_or_deployment_alias}}.es.{{private_hosted_zone_domain_name}}:443

The settings xpack.fleet.agents.fleet_server.hosts and xpack.fleet.outputs that are needed to enable this configuration in Kibana are not available in the Kibana settings in Elastic Cloud.

After you create your private connection policy, you can edit it, remove it from your deployment, or delete it.

You can edit a policy's name, description, VPC endpoint ID, and more.

1. Log in to the Elastic Cloud Console.
2. From any deployment or project on the home page, select Manage.
3. From the left navigation menu, select Access and security > Network security.

1. Find the policy you want to edit, then click the Edit button.
2. Click Update to save your changes.

If you want to a specific policy from a deployment, or delete the policy, then you need to disconnect it from any associated deployments first. You can do this from the policy's settings, or from your deployment's settings. To remove an association through the UI:

1. Log in to the Elastic Cloud Console.
2. From any deployment or project on the home page, select Manage.
3. From the left navigation menu, select Access and security > Network security.

5. Find the policy you want to edit, then click the Edit button.
6. Under Apply to resources, click the x beside the resource that you want to disconnect.
7. Click Update to save your changes.

If you need to remove a policy, you must first remove any associations with deployments.

To delete a policy:

1. Log in to the Elastic Cloud Console.
2. From any deployment or project on the home page, select Manage.
3. From the left navigation menu, select Access and security > Network security.

4. Find the policy you want to edit, then click the Delete icon. The icon is inactive if there are deployments associated with the policy.