Set up karapace instance for datahub
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	BTullis
	Feb 11 2022, 2:59 PM

Description

We need to use Karapace for the DataHub MVP.
Hopefully we won't have to use it for long, because the dependencey might be removed by upstream.

However in the meantime we have decided to deploy a single VM in Ganeti to run this service.

Details

Subject	Repo	Branch	Lines +/-
karapace: switch karapace to use kafka-jumbo1001	operations/puppet	production	+1 -1
kafka: allow access to jumbo from karapace1001	operations/puppet	production	+1 -0
karapace: add karapace role	operations/puppet	production	+128 -1
Add boot configuration for karapace1001	operations/puppet	production	+12 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	BTullis	T299910 Data Catalog MVP
Resolved	• razzi	T301562 Set up karapace instance for datahub
Resolved	akosiaris	T301563 eqiad: 1 VM requested for karapace
Resolved	• razzi	T301565 Create debian package of karapace

Event Timeline

BTullis created this task.Feb 11 2022, 2:59 PM

BTullis added a subtask: T301563: eqiad: 1 VM requested for karapace.Feb 11 2022, 3:02 PM

Assigning to @razzi if that's OK.

BTullis removed a project: Epic.Feb 11 2022, 3:10 PM

BTullis moved this task from Backlog to In Progress on the Data-Catalog board.

odimitrijevic moved this task from Incoming (new tickets) to Security & Governance on the Data-Engineering board.Feb 14 2022, 4:36 PM

• EChetty moved this task from In Progress to Next Up on the Data-Catalog board.Feb 28 2022, 4:57 PM

I have created the VM in T301563 - Now proceeding to boot the machine and make a suitable role.

BTullis closed subtask T301563: eqiad: 1 VM requested for karapace as Resolved.Mar 9 2022, 4:33 PM

Change 769468 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Add boot configuration for karapace1001

https://gerrit.wikimedia.org/r/769468

gerritbot added a project: Patch-For-Review.Mar 9 2022, 4:47 PM

Change 769468 merged by Btullis:

[operations/puppet@production] Add boot configuration for karapace1001

https://gerrit.wikimedia.org/r/769468

Maintenance_bot removed a project: Patch-For-Review.Mar 9 2022, 6:10 PM

Mentioned in SAL (#wikimedia-analytics) [2022-03-09T18:19:28Z] <btullis> btullis@ganeti1024:~$ sudo gnt-instance start karapace1001.eqiad.wmnet (T301562)

Booted karapace1001 into insetup role.

The karapace1001.eqiad.wmnet machine has now booted and is ready for karapace to be installed.

• EChetty reassigned this task from BTullis to • razzi.Mar 14 2022, 4:00 PM

BTullis moved this task from Next Up to In Progress on the Data-Catalog board.Mar 15 2022, 10:09 AM

Change 771419 had a related patch set uploaded (by Razzi; author: Razzi):

[operations/puppet@production] karapace: add karapace role

https://gerrit.wikimedia.org/r/771419

gerritbot added a project: Patch-For-Review.Mar 16 2022, 6:16 PM

• razzi moved this task from In Progress to In Review on the Data-Catalog board.Mar 16 2022, 7:07 PM

Change 771419 merged by Razzi:

[operations/puppet@production] karapace: add karapace role

https://gerrit.wikimedia.org/r/771419

• razzi added a project: Data-Engineering-Kanban.Mar 22 2022, 4:07 PM

• razzi moved this task from Next Up to In Progress on the Data-Engineering-Kanban board.

Maintenance_bot removed a project: Patch-For-Review.Mar 22 2022, 4:10 PM

The server is running; hostname is karapace1001.eqiad.wmnet and port is 8081.

curl karapace1001.eqiad.wmnet:8081
{}

Looks like the monitoring is working as well:

journalctl -u karapace
...
Mar 22 19:26:29 karapace1001 python[37435]: aiohttp.access              MainThread        INFO            0.001872s - "GET / HTTP/1.0" 200 "check_http/v2.2 (monitoring-plugins 2.2)" response=379b request_body=-b

Currently this is connected to kafka-test1006.eqiad.wmnet. It is my understanding we will use the "jumbo" kafka cluster. The following netcat times out; we'll need to open the firewall for traffic to kafka-jumbo.

nc -vz kafka-jumbo1001.eqiad.wmnet 9092
nc: connect to kafka-jumbo1001.eqiad.wmnet (10.64.0.175) port 9092 (tcp) timed out: Operation now in progress

Milimetric moved this task from In Progress to In Code Review on the Data-Engineering-Kanban board.Mar 23 2022, 4:06 PM

Merged my latest patch to remove Type=notify (https://gerrit.wikimedia.org/r/c/operations/puppet/+/773387, I used the other karapace ticket so it didn't post here) and after manually restarting karapace, puppet competes without error.

• EChetty moved this task from In Review to Done on the Data-Catalog board.Mar 28 2022, 3:56 PM

Change 774538 had a related patch set uploaded (by Razzi; author: Razzi):

[operations/puppet@production] kafka: allow access to jumbo from karapace1001

https://gerrit.wikimedia.org/r/774538

gerritbot added a project: Patch-For-Review.Mar 28 2022, 7:34 PM

Change 774538 merged by Razzi:

[operations/puppet@production] kafka: allow access to jumbo from karapace1001

https://gerrit.wikimedia.org/r/774538

• razzi moved this task from In Code Review to Ready to Deploy on the Data-Engineering-Kanban board.Mar 30 2022, 4:14 PM

Can we close this task now, or is there anything still outstanding?

BTullis closed this task as Resolved.Apr 26 2022, 2:02 PM

BTullis moved this task from Ready to Deploy to Done on the Data-Engineering-Kanban board.