Commit Graph

68 Commits

Author SHA1 Message Date
galal-hussein
c57c7646b9 Update peer address when running cluster-reset
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2021-10-25 15:49:16 -07:00
Brian Downs
9c5236189a [Release-1.20] - Add etcd s3 timeout (#4207) (#4227) 2021-10-18 10:45:29 -07:00
Brian Downs
2e9c9e8856 set transport to skip verify if se skip flag passed (#4102) (#4105) 2021-10-07 12:47:24 -07:00
Malte Starostik
5f042d1010 Fix URL pruning when joining an etcd member
* Fix URL pruning when joining an etcd member

Problem:
Existing member clientURLs were checked if they contain the joining
node's IP. In some edge cases this would prune valid URLs when the
joining IP is a substring match of the only existing member's IP.
Because of this, it was impossible to e.g. join 10.0.0.2 to an existing
node that has an IP of 10.0.0.2X or 10.0.0.2XX:

level=fatal msg="starting kubernetes: preparing server: start managed database:
joining etcd cluster: etcdclient: no available endpoints"

Solution:
Fixed by properly parsing the URLs and comparing the IPs for equality
instead of substring match.

Signed-off-by: Malte Starostik <info@stellaware.de>
(cherry picked from commit b23955e835)
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2021-08-13 11:49:45 -07:00
Brian Downs
6c39336afc account for an s3 folder when listing objects (#3807) (#3813)
* account for an s3 folder when listing objects
2021-08-09 17:34:57 -07:00
Hussein Galal
a13c952b33 [Release-1.20] fix Node stuck at deletion (#3776)
* fix Node stuck at deletion

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix Node stuck at deletion

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2021-08-06 21:20:48 +02:00
Derek Nola
6c6adcb5eb Fix to allow prune to correctly cleanup custom named snapshots (#3649) (#3673)
Signed-off-by: dereknola <derek.nola@suse.com>
2021-07-20 10:03:36 -07:00
Hussein Galal
e45dea3b14 Fix delete issue for etcd s3 snapshots (#3647)
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2021-07-16 01:34:39 +02:00
Brian Downs
5b791c1387 Backport issue 3414 release 1.20 2 (#3623)
* Add the ability to list etcd snapshots (#3303)
* add ability to list local and s3 etcd snapshots
* Add the ability to prune etcd snapshots (#3310)
* add prune subcommand to force rentention policy enforcement
* add etcd snapshot save subcommand
* add retention default and wire in s3 prune
2021-07-13 06:47:54 -07:00
Brian Downs
90e8156910 err func call
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 11:33:15 -07:00
Brian Downs
d4ca53e426 reference node name when needed
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:19:50 -07:00
Brian Downs
67bf031818 update
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:08:48 -07:00
Brian Downs
c814c05c76 update comment
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:08:32 -07:00
Brian Downs
9276c122cd remove debug line
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:08:25 -07:00
Brian Downs
ea2b314272 pr remediations
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:08:18 -07:00
Brian Downs
2fff8da520 remove debug line
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:08:12 -07:00
Brian Downs
7e6778efe5 remove unnecessary code
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:08:04 -07:00
Brian Downs
49073cd2ff add timeouts around s3 operations
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:07:55 -07:00
Brian Downs
605473e259 fix delete logic, update subcommand functionality
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:07:41 -07:00
Brian Downs
bfea9247cf initial
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 10:07:33 -07:00
Brian Downs
80b1ca2586 move map nil check
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:53:58 -07:00
Brian Downs
e9bc6bdc09 initialize map if nil
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:53:52 -07:00
Brian Downs
59b52a157d refactor snapshot storage processing logic
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:53:43 -07:00
Brian Downs
b60ee806f5 add nodeName to etcd struct, update refs
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:53:13 -07:00
Brian Downs
a1220f7fc7 Add ability for node to save etcd snapshot state at start.
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:52:49 -07:00
Brian Downs
a74fb99037 Update snapshot local and S3 logic and supporting code.
* Add the collection of the snapshots to the retry logic.
* Remove node specific entries and collected entries to maps and
  update the config map.
* Add node-name flag to the etcd-snapshot command and set in ENV.

Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:52:16 -07:00
Brian Downs
86ecb79716 remove dirs from snapshot listing
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:52:11 -07:00
Brian Downs
2ca14a331e remove unnecessary prefix:
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:52:04 -07:00
Brian Downs
62bbef7253 update struct field doc comment and get basename on s3 obj
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:51:49 -07:00
Brian Downs
83ae76fe8d update data structure and account for possible race in start
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:51:42 -07:00
Brian Downs
4c57fad5a1 update doc comment
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:51:35 -07:00
Brian Downs
297d80a924 remove unnecessary error condition
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:51:27 -07:00
Brian Downs
4e90846599 add node name to s3 structure data
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:51:20 -07:00
Brian Downs
a831b8e611 add location comment
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:51:13 -07:00
Brian Downs
d6378e5901 add addtional log entry
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:51:06 -07:00
Brian Downs
77caab3c34 update retention
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:50:58 -07:00
Brian Downs
b3e84c8862 code clean up, simplification, etc.
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:50:52 -07:00
Brian Downs
9882eb0887 update vendor and config map name
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:50:30 -07:00
Brian Downs
a6f64cc73d update node name in s3 files
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:48:54 -07:00
Brian Downs
10f0a7908a add configmap save logic
Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:48:45 -07:00
Brian Downs
977d7923dd Add ability to list snapshots.
This adds the ability to list snapshots stored locally and in S3 and
their respective metadata.

Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-06-29 09:48:38 -07:00
Brian Downs
1521e3b085 Resolve local retention issue when S3 in use.
Remove early return preventing local retention policy to be enforced
resulting in N number of snapshots being stored.

Signed-off-by: Brian Downs <brian.downs@gmail.com>
2021-04-14 12:11:32 -07:00
Brian Downs
0167f86f5d Have Bootstrap Data Stored in etcd at Completed Start (#3038)
* have state stored in etcd at completed start and remove unneeded code

Signed-off-by: Brian Downs <brian.downs@gmail.com>
(cherry picked from commit 7c99f8645d)
2021-03-15 16:04:10 -07:00
Brian Downs
1da86eaa0b Etcd Snapshot/Restore to/from S3 Compatible Backends (#2902)
* Add functionality for etcd snapshot/restore to and from S3 compatible backends.
* Update etcd restore functionality to extract and write certificates and configs from snapshot.

Signed-off-by: Brian Downs <brian.downs@gmail.com>
(cherry picked from commit 4d1f9eda9d)
2021-03-15 16:04:04 -07:00
Hussein Galal
b0c0d372b8 [release-1.20] Add disable components flags (#3019)
* Add disable flags for control components (#2900)

* Add disable flags to control components

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* golint

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* more fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fixes to disable flags

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Add comments to functions

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Fix joining problem

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* more fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* golint

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix ticker

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix role labels

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* more fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* remove etcd member if disable etcd is passed

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* update dynamiclistener

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* change error to warn when removing self from etcd members

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Mark disable components flags as experimental

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2021-03-05 00:04:48 +02:00
Brad Davidson
071de833ae Fix typo in field tag
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2021-01-22 19:38:37 -08:00
Yuriy
06fda7accf Add functionality to bind custom IP address for Etcd metrics endpoint (#2750)
* Add functionality to bind custom IP address for Etcd metrics endpoint

Signed-off-by: yuriydzobak <yurii.dzobak@lotusflare.com>
2021-01-22 17:40:48 -08:00
Brian Downs
13229019f8 Add ability to perform an etcd on-demand snapshot via cli (#2819)
* add ability to perform an etcd on-demand snapshot via cli
2021-01-21 14:09:15 -07:00
MonzElmasry
86f68d5d62 change etcd dir permission if it exists
Signed-off-by: MonzElmasry <menna.elmasry@rancher.com>
2021-01-08 23:47:36 +02:00
Brad Davidson
8e4d3e645b Restore legacy master role for etcd nodes
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2020-12-15 15:15:46 -08:00