OLD | NEW |
1 # -*- coding: utf-8 -*- | 1 # -*- coding: utf-8 -*- |
2 # Copyright 2014 Google Inc. All Rights Reserved. | 2 # Copyright 2014 Google Inc. All Rights Reserved. |
3 # | 3 # |
4 # Licensed under the Apache License, Version 2.0 (the "License"); | 4 # Licensed under the Apache License, Version 2.0 (the "License"); |
5 # you may not use this file except in compliance with the License. | 5 # you may not use this file except in compliance with the License. |
6 # You may obtain a copy of the License at | 6 # You may obtain a copy of the License at |
7 # | 7 # |
8 # http://www.apache.org/licenses/LICENSE-2.0 | 8 # http://www.apache.org/licenses/LICENSE-2.0 |
9 # | 9 # |
10 # Unless required by applicable law or agreed to in writing, software | 10 # Unless required by applicable law or agreed to in writing, software |
(...skipping 184 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
195 gsutil -m rsync -r -d -n gs://your-bucket/data gs://your-bucket | 195 gsutil -m rsync -r -d -n gs://your-bucket/data gs://your-bucket |
196 | 196 |
197 it will be immediately evident that running that command without the -n | 197 it will be immediately evident that running that command without the -n |
198 option would cause many objects to be deleted. | 198 option would cause many objects to be deleted. |
199 | 199 |
200 2. Enable object versioning in your bucket, which will allow you to restore | 200 2. Enable object versioning in your bucket, which will allow you to restore |
201 objects if you accidentally delete them. For more details see | 201 objects if you accidentally delete them. For more details see |
202 "gsutil help versions". | 202 "gsutil help versions". |
203 | 203 |
204 | 204 |
| 205 <B>BE CAREFUL WHEN SYNCHRONIZING OVER SYMLINKS</B> |
| 206 If you run gsutil rsync over a directory containing symlinks, the links will |
| 207 be followed. Hence, a command like: |
| 208 |
| 209 gsutil rsync -r ./dir gs://my-bucket |
| 210 |
| 211 will create objects in my-bucket containing the data from the files to which |
| 212 the symlinks point. In most cases this is not the behavior you would want. A |
| 213 couple of examples of how this can cause problems: |
| 214 |
| 215 * If you use gsutil rsync as a simple way to backup a directory to a bucket, |
| 216 restoring from that bucket will result in files where the symlinks used |
| 217 to be. At best this is wasteful of space, and at worst it can result in |
| 218 outdated data or broken applications -- depending on what is consuming |
| 219 the symlinks. |
| 220 |
| 221 * If you use gsutil rsync over directories containing broken symlinks, |
| 222 gsutil rsync will abort (unless you pass the -e option). |
| 223 |
| 224 * gsutil rsync skips symlinks that point to directories. |
| 225 |
| 226 Since gsutil is meant to support data operations (like moving a data set to |
| 227 the cloud for computational processing) and needs to be compatible both in |
| 228 the cloud and across common operating systems, there are no plans for gsutil |
| 229 rsync to support symlinks. |
| 230 |
| 231 We recommend that customers either: |
| 232 ·· |
| 233 * not use gstil rsync over directories containing symlinks; |
| 234 * use the -e option to exclude symlinks; or, |
| 235 * use a tool (such as tar) that preserves symlinks and package up |
| 236 directories containing symlinks before uploading to the cloud. |
| 237 |
| 238 |
205 <B>EVENTUAL CONSISTENCY WITH NON-GOOGLE CLOUD PROVIDERS</B> | 239 <B>EVENTUAL CONSISTENCY WITH NON-GOOGLE CLOUD PROVIDERS</B> |
206 While Google Cloud Storage is strongly consistent, some cloud providers | 240 While Google Cloud Storage is strongly consistent, some cloud providers |
207 only support eventual consistency. You may encounter scenarios where rsync | 241 only support eventual consistency. You may encounter scenarios where rsync |
208 synchronizes using stale listing data when working with these other cloud | 242 synchronizes using stale listing data when working with these other cloud |
209 providers. For example, if you run rsync immediately after uploading an | 243 providers. For example, if you run rsync immediately after uploading an |
210 object to an eventually consistent cloud provider, the added object may not | 244 object to an eventually consistent cloud provider, the added object may not |
211 yet appear in the provider’s listing. Consequently, rsync will miss adding | 245 yet appear in the provider’s listing. Consequently, rsync will miss adding |
212 the object to the destination. If this happens you can rerun the rsync | 246 the object to the destination. If this happens you can rerun the rsync |
213 operation again later (after the object listing has "caught up"). | 247 operation again later (after the object listing has "caught up"). |
214 | 248 |
(...skipping 1371 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
1586 if gzip_arg_exts and gzip_arg_all: | 1620 if gzip_arg_exts and gzip_arg_all: |
1587 raise CommandException( | 1621 raise CommandException( |
1588 'Specifying both the -j and -J options together is invalid.') | 1622 'Specifying both the -j and -J options together is invalid.') |
1589 self.gzip_encoded = gzip_encoded | 1623 self.gzip_encoded = gzip_encoded |
1590 self.gzip_exts = gzip_arg_exts or gzip_arg_all | 1624 self.gzip_exts = gzip_arg_exts or gzip_arg_all |
1591 | 1625 |
1592 return CreateCopyHelperOpts( | 1626 return CreateCopyHelperOpts( |
1593 canned_acl=canned_acl, | 1627 canned_acl=canned_acl, |
1594 preserve_acl=self.preserve_acl, | 1628 preserve_acl=self.preserve_acl, |
1595 skip_unsupported_objects=self.skip_unsupported_objects) | 1629 skip_unsupported_objects=self.skip_unsupported_objects) |
OLD | NEW |