marathon-lb配置及nginx负载

发表于 | 分类于 | | 游览

marathon-lb配置

marathon-lb get images

Marathon-lb既是一个服务发现工具，也是负载均衡工具，它集成了haproxy，自动获取各个app的信息，为每一组app生成haproxy配置，通过servicePort或者web虚拟主机提供服务。

要使用marathonn-lb，每组app必须设置HAPROXY_GROUP标签。

Marathon-lb运行时绑定在各组app定义的服务端口（servicePort，如果app不定义servicePort，marathon会随机分配端口号）上，可以通过marathon-lb所在节点的相关服务端口访问各组app。

例如：marathon-lb部署在slave5，test-app 部署在slave1，test-app 的servicePort是10004，那么可以在slave5的 10004端口访问到test-app提供的服务。

由于servicePort 非80、443端口（80、443端口已被marathon-lb中的 haproxy独占），对于web服务来说不太方便，可以使用 haproxy虚拟主机解决这个问题：

在提供web服务的app配置里增加HAPROXY_{n}_VHOST（WEB虚拟主机）标签，marathon-lb会自动把这组app的WEB集群服务发布在marathon-lb所在节点的80和443端口上，用户设置DNS后通过虚拟主机名来访问。

官方下载镜像

1
2
3
4
5
6
7
images url :
https://store.docker.com/community/images/mesosphere/marathon-lb

docker pull mesosphere/marathon-lb

github url:
https://github.com/mesosphere/marathon-lb

运行

docker

1
docker run -d --privileged -e PORTS=9090 --net=host docker.io/mesosphere/marathon-lb sse -m http://marathon1_ip:8080 -m http://marathon2_ip:8080 -m http://master3_ip:8080  --group external

marathon

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
vim marathon-lb.json

{
    "id": "marathon-lb-testv1",
    "instances": 1,
    "constraints": [
        [
            "hostname",
            "CLUSTER",
            "host-hostname.com"
        ]
    ],
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "docker.io/mesosphere/marathon-lb:latest",
            "privileged": true,
            "network": "HOST"
        }
    },
    "args": [
        "sse",
        "-m",
        "http://10.10.131.78:8080",
        "--auth-credentials",
        "admin:adminpassword",
        "--group",
        "external"
    ]
}


curl -X POST http://10.10.131.78:8080/v2/apps -d @marathon-lb.json -H "Content-type: application/json"

marathon-lb API

Endpoint Description
:9090/haproxy?stats HAProxy stats endpoint. This produces an HTML page which can be viewed in your browser, providing various statistics about the current HAProxy instance.
:9090/haproxy?stats;csv This is a CSV version of the stats above, which can be consumed by other tools. For example, it’s used in the zdd.py script.
:9090/_haproxy_health_check HAProxy health check endpoint. Returns 200 OK if HAProxy is healthy.
:9090/_haproxy_getconfig Returns the HAProxy config file as it was when HAProxy was started. Implemented in getconfig.lua.
:9090/_haproxy_getvhostmap Returns the HAProxy vhost to backend map. This endpoint returns HAProxy map file only when the –haproxy-map flag is enabled, it returns an empty string otherwise. Implemented in getmaps.lua.
:9090/_haproxy_getappmap Returns the HAProxy app ID to backend map. Like _haproxy_getvhostmap, this requires the –haproxy-map flag to be enabled and returns an empty string otherwise. Also implemented in getmaps.lua.
:9090/_haproxy_getpids Returns the PIDs for all HAProxy instances within the current process namespace. This literally returns $(pidof haproxy). Implemented in getpids.lua. This is also used by the zdd.py script to determine if connections have finished draining during a deploy.
:9090/_mlb_signal/hup* Sends a SIGHUP signal to the marathon-lb process, causing it to fetch the running apps from Marathon and reload the HAProxy config as though an event was received from Marathon.
:9090/_mlb_signal/usr1* Sends a SIGUSR1 signal to the marathon-lb process, causing it to restart HAProxy with the existing config, without checking Marathon for changes.

如常用： http://marathon-lb-ip:9090/haproxy?stats

nginx start

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# vim nginx.json

{
    "id": "nginx-test",
    "cpus": 0.2,
    "mem": 128,
    "instances": 1,
  "labels": {
     "HAPROXY_GROUP":"external"
     "HAPROXY_0_VHOST":"nginx.test.com"
  },
    "uris": [
        "http://10.10.130.201/download/docker_img/db-harbor-admin.tar.gz"
    ],
    "healthChecks": [{ "path": "/" }],
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "nginx:1.13",
            "network": "BRIDGE",
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 0,
                    "servicePort": 10000,
                    "protocol": "tcp"
                }
            ]
        }
    }
}

# curl -X POST http://10.10.131.78:8080/v2/apps -d @nginx.json -H "Content-type: application/json"

说明：

  1. 一定要加上HAPROXY_GROUP标签，它填写的是marathon-lb创建时定义的组名
  2. HAPROXY_0_VHOST是标签名，对于web服务可以加上VHOST标签，让marathon-lb设置WEB虚拟主机；
  3. containerPort为80,是指容器内的端口。
  4. hostPort是当前主机映射到contenterPort的端口，如果hostPort为0的话,则说明是随机的。
  5. serverPort是marathon-lb需要配置的haproxy代理暴露的端口,这里设置为10000，说明访问marathon-lb机器的10000端口就可为访问这个应用容器的80端口。

访问marathon-lb

ip 访问

1
curl http://marathon-lb_ip:10000/
  • 访问marathon-lb部署的宿主机ip地址和serverPort的端口。

域名访问

1
2
3
4
5
6
7
8
需要添加dns解析，根据 "HAPROXY_0_VHOST":"nginx.test.com" 设置的配置。
如：
vim /etc/hosts  添加
10.10.131.151		nginx.test.com

这里 10.10.131.151 是 marathon-lb 的ip地址

curl nginx.test.com  即可

marathon-lb 代理80端口

默认marathon-lb 80和443端口是被占用的，所以nginx在发布的时候“serverPort”是不能设置为80和443端口的。

为了解决这个问题，需要更改源码，重新生成镜像。

首先现在 marathon-lb源码：

1
2
3
4
5
6
7
8
9
10
11
# git clone https://github.com/mesosphere/marathon-lb.git
# cd marathon-lb

在这个目录下找到所有80、443端口信息。改为其他端口

# grep 80 . -R
找到相应文件，80 替换为7080
:%s/80/7080/g

找到相应文件，443 替换为7443
:%s/443/7443/g

重新生成镜像

1
2
3
docker build -t marathon-lb-7080 .

成功之后 docker images 就会多出 marathon-lb-7080 镜像

感觉文章还可以的话，帮忙点点下面的广告哦！ 谢谢支持！

感谢您的支持！