author | Pierre-Yves David <pierre-yves.david@octobus.net> |
Sat, 01 Feb 2020 12:50:10 +0100 | |
changeset 5111 | 1fe3f7ffb462 |
parent 5047 | a8ac174010a5 |
child 5193 | a4d081923c81 |
permissions | -rw-r--r-- |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
1 |
# Extension to provide automatic caching of bundle server for pull |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
2 |
# |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
3 |
# Copyright 2018 Pierre-Yves David <pierre-yves.david@ens-lyon.org> |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
4 |
# |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
5 |
# This software may be used and distributed according to the terms of the |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
6 |
# GNU General Public License version 2 or any later version. |
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
7 |
"""pullbundle: automatic server side bundle caching |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
8 |
|
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
9 |
General principle |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
10 |
================= |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
11 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
12 |
This extension provides a means for server to use pre-computed bundle for |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
13 |
serving arbitrary pulls. If missing, the necessary pre-computed bundle will be |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
14 |
generated on demand. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
15 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
16 |
To maximize usage of existing cached bundle, each pull will be served through |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
17 |
multiple bundles. The bundle will be created using "standard range" from the |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
18 |
"stablerange" principle. The "stablerange" concept if already used for |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
19 |
obsmarkers discovery in the evolve extensions. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
20 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
21 |
Using pull Bundle |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
22 |
================= |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
23 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
24 |
All configuration is only required server side. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
25 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
26 |
The "stablerange" code currently still live in the evolve extensions, so for |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
27 |
now enabling that extensions is required: |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
28 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
29 |
You need at minimum the following configuration: |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
30 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
31 |
[extensions] |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
32 |
evolve=yes |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
33 |
pullbundle=yes |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
34 |
[experimental] |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
35 |
obshashrange.warm-cache = yes |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
36 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
37 |
If you do not want to use evolution server side, you should disable obsmarkers exchange: |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
38 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
39 |
[experimental] |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
40 |
evolution.exchange=no |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
41 |
|
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
42 |
Extra Configuration |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
43 |
=================== |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
44 |
|
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
45 |
[pullbundle] |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
46 |
# By default bundles are stored `.hg/cache/pullbundles/. |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
47 |
# This can be changed with the following config: |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
48 |
cache-directory=/absolute/path |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
49 |
|
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
50 |
Implementation status |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
51 |
===================== |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
52 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
53 |
Both for stablerange and pullbundle use "simple" initial implementations. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
54 |
Theses implemenations focus on testing the algorithms and proving the features |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
55 |
works. Yet they are already useful and used in production. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
56 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
57 |
Performances are expected to greatly improved in the final implementation, |
4141
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
58 |
especially if some of it end up being compiled code. |
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
59 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
60 |
This first implementation lacks the ability to server the cached bundle from a |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
61 |
CDN. We'll want this limitation to be lifted quickly. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
62 |
|
4141
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
63 |
The way mercurial core report progress is designed for the receival of a single |
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
64 |
changegroup. So currently using pullbundle means flooding the user with output. |
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
65 |
This will have to be fixed. |
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
66 |
|
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
67 |
Why is does this live in the same repository as evolve |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
68 |
====================================================== |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
69 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
70 |
There is no fundamental reasons for live in the same repository. However, the |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
71 |
stablerange data-structure lives in evolve, so it was simpler to put this new |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
72 |
extensions next to it. As soon as stable range have been upstreamed, we won't |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
73 |
need the dependency to the evolve extension anymore. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
74 |
""" |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
75 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
76 |
import collections |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
77 |
import errno |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
78 |
import random |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
79 |
import os |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
80 |
|
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
81 |
from mercurial import ( |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
82 |
changegroup, |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
83 |
discovery, |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
84 |
error, |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
85 |
exchange, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
86 |
narrowspec, |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
87 |
node as nodemod, |
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
88 |
registrar, |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
89 |
scmutil, |
5044
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
90 |
ui as uimod, |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
91 |
util, |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
92 |
) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
93 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
94 |
from mercurial.i18n import _ |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
95 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
96 |
__version__ = b'0.1.1' |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
97 |
testedwith = b'4.4 4.5 4.6 4.7.1' |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
98 |
minimumhgversion = b'4.4' |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
99 |
buglink = b'https://bz.mercurial-scm.org/' |
4135
47f1d7b4305d
pullbundle: add basic extension metadata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4134
diff
changeset
|
100 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
101 |
cmdtable = {} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
102 |
command = registrar.command(cmdtable) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
103 |
|
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
104 |
configtable = {} |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
105 |
configitem = registrar.configitem(configtable) |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
106 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
107 |
configitem(b'pullbundle', b'cache-directory', |
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
108 |
default=None, |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
109 |
) |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
110 |
|
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
111 |
# generic wrapping |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
112 |
|
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
113 |
def uisetup(ui): |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
114 |
exchange.getbundle2partsmapping[b'changegroup'] = _getbundlechangegrouppart |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
115 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
116 |
def _getbundlechangegrouppart(bundler, repo, source, bundlecaps=None, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
117 |
b2caps=None, heads=None, common=None, **kwargs): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
118 |
"""add a changegroup part to the requested bundle""" |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
119 |
if not kwargs.get(r'cg', True): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
120 |
return |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
121 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
122 |
version = b'01' |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
123 |
cgversions = b2caps.get(b'changegroup') |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
124 |
if cgversions: # 3.1 and 3.2 ship with an empty value |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
125 |
cgversions = [v for v in cgversions |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
126 |
if v in changegroup.supportedoutgoingversions(repo)] |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
127 |
if not cgversions: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
128 |
raise ValueError(_(b'no common changegroup version')) |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
129 |
version = max(cgversions) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
130 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
131 |
outgoing = exchange._computeoutgoing(repo, heads, common) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
132 |
if not outgoing.missing: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
133 |
return |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
134 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
135 |
if kwargs.get(r'narrow', False): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
136 |
include = sorted(filter(bool, kwargs.get(r'includepats', []))) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
137 |
exclude = sorted(filter(bool, kwargs.get(r'excludepats', []))) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
138 |
filematcher = narrowspec.match(repo.root, include=include, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
139 |
exclude=exclude) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
140 |
else: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
141 |
filematcher = None |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
142 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
143 |
# START OF ALTERED PART |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
144 |
makeallcgpart(bundler.newpart, repo, outgoing, version, source, bundlecaps, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
145 |
filematcher, cgversions) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
146 |
# END OF ALTERED PART |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
147 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
148 |
if kwargs.get(r'narrow', False) and (include or exclude): |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
149 |
narrowspecpart = bundler.newpart(b'narrow:spec') |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
150 |
if include: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
151 |
narrowspecpart.addparam( |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
152 |
b'include', b'\n'.join(include), mandatory=True) |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
153 |
if exclude: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
154 |
narrowspecpart.addparam( |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
155 |
b'exclude', b'\n'.join(exclude), mandatory=True) |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
156 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
157 |
def makeallcgpart(newpart, repo, outgoing, version, source, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
158 |
bundlecaps, filematcher, cgversions): |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
159 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
160 |
pullbundle = not filematcher |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
161 |
if pullbundle and not util.safehasattr(repo, 'stablerange'): |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
162 |
repo.ui.warn(b'pullbundle: required extension "evolve" are missing, skipping pullbundle\n') |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
163 |
pullbundle = False |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
164 |
if filematcher: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
165 |
makeonecgpart(newpart, repo, None, outgoing, version, source, bundlecaps, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
166 |
filematcher, cgversions) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
167 |
else: |
4133
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
168 |
start = util.timer() |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
169 |
slices = sliceoutgoing(repo, outgoing) |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
170 |
end = util.timer() |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
171 |
msg = _(b'pullbundle-cache: "missing" set sliced into %d subranges ' |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
172 |
b'in %f seconds\n') |
4133
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
173 |
repo.ui.write(msg % (len(slices), end - start)) |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
174 |
for sliceid, sliceout in slices: |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
175 |
makeonecgpart(newpart, repo, sliceid, sliceout, version, source, bundlecaps, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
176 |
filematcher, cgversions) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
177 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
178 |
# stable range slicing |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
179 |
|
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
180 |
DEBUG = False |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
181 |
|
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
182 |
def sliceoutgoing(repo, outgoing): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
183 |
cl = repo.changelog |
4929
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
184 |
rev = getgetrev(cl) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
185 |
node = cl.node |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
186 |
revsort = repo.stablesort |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
187 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
188 |
missingrevs = set(rev(n) for n in outgoing.missing) |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
189 |
if DEBUG: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
190 |
ms = missingrevs.copy() |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
191 |
ss = [] |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
192 |
allslices = [] |
4147
88e922eca4e2
pullbundle: deal with head in a predicable order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4145
diff
changeset
|
193 |
missingheads = [rev(n) for n in sorted(outgoing.missingheads, reverse=True)] |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
194 |
for head in missingheads: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
195 |
localslices = [] |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
196 |
localmissing = set(repo.revs(b'%ld and ::%d', missingrevs, head)) |
4149
4abec3a1208f
pullbundle: fix style warning
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4148
diff
changeset
|
197 |
thisrunmissing = localmissing.copy() |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
198 |
while localmissing: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
199 |
slicerevs = [] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
200 |
for r in revsort.walkfrom(repo, head): |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
201 |
if r not in thisrunmissing: |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
202 |
break |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
203 |
slicerevs.append(r) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
204 |
slicenodes = [node(r) for r in slicerevs] |
4140
9b71aa222f8e
pullbundle: issue local branches in the right order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4139
diff
changeset
|
205 |
localslices.append(canonicalslices(repo, slicenodes)) |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
206 |
if DEBUG: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
207 |
ss.append(slicerevs) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
208 |
missingrevs.difference_update(slicerevs) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
209 |
localmissing.difference_update(slicerevs) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
210 |
if localmissing: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
211 |
heads = list(repo.revs(b'heads(%ld)', localmissing)) |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
212 |
heads.sort(key=node) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
213 |
head = heads.pop() |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
214 |
if heads: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
215 |
thisrunmissing = repo.revs(b'%ld and only(%d, %ld)', |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
216 |
localmissing, |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
217 |
head, |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
218 |
heads) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
219 |
else: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
220 |
thisrunmissing = localmissing.copy() |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
221 |
if DEBUG: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
222 |
for s in reversed(ss): |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
223 |
ms -= set(s) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
224 |
missingbase = repo.revs(b'parents(%ld) and %ld', s, ms) |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
225 |
if missingbase: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
226 |
repo.ui.write_err(b'!!! rev bundled while parents missing\n') |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
227 |
repo.ui.write_err(b' parent: %s\n' % list(missingbase)) |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
228 |
pb = repo.revs(b'%ld and children(%ld)', s, missingbase) |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
229 |
repo.ui.write_err(b' children: %s\n' % list(pb)) |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
230 |
h = repo.revs(b'heads(%ld)', s) |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
231 |
repo.ui.write_err(b' heads: %s\n' % list(h)) |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
232 |
raise error.ProgrammingError(b'issuing a range before its parents') |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
233 |
|
4140
9b71aa222f8e
pullbundle: issue local branches in the right order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4139
diff
changeset
|
234 |
for s in reversed(localslices): |
9b71aa222f8e
pullbundle: issue local branches in the right order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4139
diff
changeset
|
235 |
allslices.extend(s) |
4136
be3a94d3105f
pullbundle: attempt to save stablerange cache after each computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4135
diff
changeset
|
236 |
# unknown subrange might had to be computed |
be3a94d3105f
pullbundle: attempt to save stablerange cache after each computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4135
diff
changeset
|
237 |
repo.stablerange.save(repo) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
238 |
return [(rangeid, outgoingfromnodes(repo, nodes)) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
239 |
for rangeid, nodes in allslices] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
240 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
241 |
def canonicalslices(repo, nodes): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
242 |
depth = repo.depthcache.get |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
243 |
stablerange = repo.stablerange |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
244 |
rangelength = lambda x: stablerange.rangelength(repo, x) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
245 |
headrev = repo.changelog.rev(nodes[0]) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
246 |
nbrevs = len(nodes) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
247 |
headdepth = depth(headrev) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
248 |
skipped = headdepth - nbrevs |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
249 |
rangeid = (headrev, skipped) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
250 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
251 |
subranges = canonicalsubranges(repo, stablerange, rangeid) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
252 |
idx = 0 |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
253 |
slices = [] |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
254 |
nodes.reverse() |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
255 |
for rangeid in subranges: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
256 |
size = rangelength(rangeid) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
257 |
slices.append((rangeid, nodes[idx:idx + size])) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
258 |
idx += size |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
259 |
### slow code block to validate ranges content |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
260 |
# rev = repo.changelog.nodemap.get |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
261 |
# for ri, ns in slices: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
262 |
# a = set(rev(n) for n in ns) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
263 |
# b = set(repo.stablerange.revsfromrange(repo, ri)) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
264 |
# l = repo.stablerange.rangelength(repo, ri) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
265 |
# repo.ui.write('range-length: %d-%d %s %s\n' % (ri[0], ri[1], l, len(a))) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
266 |
# if a != b: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
267 |
# d = (ri[0], ri[1], b - a, a - b) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
268 |
# repo.ui.write("mismatching content: %d-%d -%s +%s\n" % d) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
269 |
return slices |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
270 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
271 |
def canonicalsubranges(repo, stablerange, rangeid): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
272 |
"""slice a size of nodes into most reusable subranges |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
273 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
274 |
We try to slice a range into a set of "largest" and "canonical" stable |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
275 |
range. |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
276 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
277 |
It might make sense to move this function as a 'stablerange' method. |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
278 |
""" |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
279 |
headrev, skip = rangeid |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
280 |
rangedepth = stablerange.depthrev(repo, rangeid[0]) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
281 |
canonicals = [] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
282 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
283 |
# 0. find the first power of 2 higher than this range depth |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
284 |
cursor = 1 |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
285 |
while cursor <= rangedepth: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
286 |
cursor *= 2 |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
287 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
288 |
# 1. find first cupt |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
289 |
precut = cut = 0 |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
290 |
while True: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
291 |
if skip <= cut: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
292 |
break |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
293 |
if cut + cursor < rangedepth: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
294 |
precut = cut |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
295 |
cut += cursor |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
296 |
if cursor == 1: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
297 |
break |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
298 |
cursor //= 2 |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
299 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
300 |
# 2. optimise, bottom part |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
301 |
if skip != cut: |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
302 |
currentsize = tailsize = cut - skip |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
303 |
assert 0 < tailsize, tailsize |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
304 |
|
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
305 |
# we need to take several "standard cut" in the bottom part |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
306 |
# |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
307 |
# This is similar to what we will do for the top part, we reusing the |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
308 |
# existing structure is a bit more complex. |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
309 |
allcuts = list(reversed(standardcut(tailsize))) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
310 |
prerange = (headrev, precut) |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
311 |
### slow code block to check we operate on the right data |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
312 |
# rev = repo.changelog.nodemap.get |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
313 |
# allrevs = [rev(n) for n in nodes] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
314 |
# allrevs.reverse() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
315 |
# prerevs = repo.stablerange.revsfromrange(repo, prerange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
316 |
# assert allrevs == prerevs[(len(prerevs) - len(allrevs)):] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
317 |
# end of check |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
318 |
sub = list(stablerange.subranges(repo, prerange)[:-1]) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
319 |
|
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
320 |
bottomranges = [] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
321 |
# XXX we might be able to reuse core stable-range logic instead of |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
322 |
# redoing this manually |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
323 |
currentrange = sub.pop() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
324 |
currentsize = stablerange.rangelength(repo, currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
325 |
currentcut = None |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
326 |
while allcuts or currentcut is not None: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
327 |
# get the next cut if needed |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
328 |
if currentcut is None: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
329 |
currentcut = allcuts.pop() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
330 |
# deal attemp a cut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
331 |
if currentsize == currentcut: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
332 |
bottomranges.append(currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
333 |
currentcut = None |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
334 |
elif currentsize < currentcut: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
335 |
bottomranges.append(currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
336 |
currentcut -= currentsize |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
337 |
else: # currentsize > currentcut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
338 |
newskip = currentrange[1] + (currentsize - currentcut) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
339 |
currentsub = stablerange._slicesrangeat(repo, currentrange, newskip) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
340 |
bottomranges.append(currentsub.pop()) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
341 |
sub.extend(currentsub) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
342 |
currentcut = None |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
343 |
currentrange = sub.pop() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
344 |
currentsize = stablerange.rangelength(repo, currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
345 |
bottomranges.reverse() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
346 |
canonicals.extend(bottomranges) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
347 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
348 |
# 3. take recursive subrange until we get to a power of two size? |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
349 |
current = (headrev, cut) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
350 |
while not poweroftwo(stablerange.rangelength(repo, current)): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
351 |
sub = stablerange.subranges(repo, current) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
352 |
canonicals.extend(sub[:-1]) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
353 |
current = sub[-1] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
354 |
canonicals.append(current) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
355 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
356 |
return canonicals |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
357 |
|
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
358 |
def standardcut(size): |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
359 |
assert 0 < size |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
360 |
# 0. find the first power of 2 higher than this range depth |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
361 |
cut = 1 |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
362 |
while cut <= size: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
363 |
cut *= 2 |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
364 |
|
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
365 |
allcuts = [] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
366 |
# 1. find all standard expected cut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
367 |
while 1 < cut and size: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
368 |
cut //= 2 |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
369 |
if cut <= size: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
370 |
allcuts.append(cut) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
371 |
size -= cut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
372 |
return allcuts |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
373 |
|
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
374 |
def poweroftwo(num): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
375 |
return num and not num & (num - 1) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
376 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
377 |
def outgoingfromnodes(repo, nodes): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
378 |
return discovery.outgoing(repo, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
379 |
missingroots=nodes, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
380 |
missingheads=nodes) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
381 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
382 |
# changegroup part construction |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
383 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
384 |
def _changegroupinfo(repo, nodes, source): |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
385 |
if repo.ui.verbose or source == b'bundle': |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
386 |
repo.ui.status(_(b"%d changesets found\n") % len(nodes)) |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
387 |
|
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
388 |
def _makenewstream(newpart, repo, outgoing, version, source, |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
389 |
bundlecaps, filematcher, cgversions): |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
390 |
old = changegroup._changegroupinfo |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
391 |
try: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
392 |
changegroup._changegroupinfo = _changegroupinfo |
4145
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
393 |
if filematcher is not None: |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
394 |
cgstream = changegroup.makestream(repo, outgoing, version, source, |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
395 |
bundlecaps=bundlecaps, |
5034
1015a1dbaf7c
pullbundle: kwarg to changegroup.makestream() is called matcher
Anton Shestakov <av6@dwimlabs.net>
parents:
4929
diff
changeset
|
396 |
matcher=filematcher) |
4145
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
397 |
else: |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
398 |
cgstream = changegroup.makestream(repo, outgoing, version, source, |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
399 |
bundlecaps=bundlecaps) |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
400 |
finally: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
401 |
changegroup._changegroupinfo = old |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
402 |
|
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
403 |
nbchanges = len(outgoing.missing) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
404 |
pversion = None |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
405 |
if cgversions: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
406 |
pversion = version |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
407 |
return (cgstream, nbchanges, pversion) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
408 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
409 |
def _makepartfromstream(newpart, repo, cgstream, nbchanges, version): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
410 |
# same as upstream code |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
411 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
412 |
part = newpart(b'changegroup', data=cgstream) |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
413 |
if version: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
414 |
part.addparam(b'version', version) |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
415 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
416 |
part.addparam(b'nbchanges', b'%d' % nbchanges, |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
417 |
mandatory=False) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
418 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
419 |
if b'treemanifest' in repo.requirements: |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
420 |
part.addparam(b'treemanifest', b'1') |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
421 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
422 |
# cache management |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
423 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
424 |
def cachedir(repo): |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
425 |
cachedir = repo.ui.config(b'pullbundle', b'cache-directory') |
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
426 |
if cachedir is not None: |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
427 |
return cachedir |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
428 |
return repo.cachevfs.join(b'pullbundles') |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
429 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
430 |
def getcache(repo, bundlename): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
431 |
cdir = cachedir(repo) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
432 |
bundlepath = os.path.join(cdir, bundlename) |
4139
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
433 |
if not os.path.exists(bundlepath): |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
434 |
return None |
4139
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
435 |
# delay file opening as much as possible this introduce a small race |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
436 |
# condition if someone remove the file before we actually use it. However |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
437 |
# opening too many file will not work. |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
438 |
|
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
439 |
def data(): |
4807
ea8da5aa23c6
python3: add raw prefix to open()-like functions
Raphaël Gomès <rgomes@octobus.net>
parents:
4756
diff
changeset
|
440 |
with open(bundlepath, r'rb') as fd: |
4139
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
441 |
for chunk in util.filechunkiter(fd): |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
442 |
yield chunk |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
443 |
return data() |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
444 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
445 |
def cachewriter(repo, bundlename, stream): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
446 |
cdir = cachedir(repo) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
447 |
bundlepath = os.path.join(cdir, bundlename) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
448 |
try: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
449 |
os.makedirs(cdir) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
450 |
except OSError as exc: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
451 |
if exc.errno == errno.EEXIST: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
452 |
pass |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
453 |
with util.atomictempfile(bundlepath) as cachefile: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
454 |
for chunk in stream: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
455 |
cachefile.write(chunk) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
456 |
yield chunk |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
457 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
458 |
BUNDLEMASK = b"%s-%s-%010iskip-%010isize.hg" |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
459 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
460 |
def makeonecgpart(newpart, repo, rangeid, outgoing, version, source, |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
461 |
bundlecaps, filematcher, cgversions): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
462 |
bundlename = cachedata = None |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
463 |
if rangeid is not None: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
464 |
nbchanges = repo.stablerange.rangelength(repo, rangeid) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
465 |
headnode = nodemod.hex(repo.changelog.node(rangeid[0])) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
466 |
# XXX do we need to use cgversion in there? |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
467 |
bundlename = BUNDLEMASK % (version, headnode, rangeid[1], nbchanges) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
468 |
cachedata = getcache(repo, bundlename) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
469 |
if cachedata is None: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
470 |
partdata = _makenewstream(newpart, repo, outgoing, version, source, |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
471 |
bundlecaps, filematcher, cgversions) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
472 |
if bundlename is not None: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
473 |
cgstream = cachewriter(repo, bundlename, partdata[0]) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
474 |
partdata = (cgstream,) + partdata[1:] |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
475 |
else: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
476 |
if repo.ui.verbose or source == b'bundle': |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
477 |
repo.ui.status(_(b"%d changesets found in caches\n") % nbchanges) |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
478 |
pversion = None |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
479 |
if cgversions: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
480 |
pversion = version |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
481 |
partdata = (cachedata, nbchanges, pversion) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
482 |
return _makepartfromstream(newpart, repo, *partdata) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
483 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
484 |
@command(b'debugpullbundlecacheoverlap', |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
485 |
[(b'', b'count', 100, _(b'of "client" pulling')), |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
486 |
(b'', b'min-cache', 1, _(b'minimum size of cached bundle')), |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
487 |
], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
488 |
_(b'hg debugpullbundlecacheoverlap [--client 100] REVSET')) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
489 |
def debugpullbundlecacheoverlap(ui, repo, *revs, **opts): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
490 |
'''Display statistic on bundle cache hit |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
491 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
492 |
This command "simulate pulls from multiple clients. Each using a random |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
493 |
subset of revisions defined by REVSET. And display statistic about the |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
494 |
overlap in bundle necessary to serve them. |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
495 |
''' |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
496 |
actionrevs = scmutil.revrange(repo, revs) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
497 |
if not revs: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
498 |
raise error.Abort(b'No revision selected') |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
499 |
count = opts['count'] |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
500 |
min_cache = opts['min_cache'] |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
501 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
502 |
bundlehits = collections.defaultdict(lambda: 0) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
503 |
pullstats = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
504 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
505 |
rlen = lambda rangeid: repo.stablerange.rangelength(repo, rangeid) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
506 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
507 |
repo.ui.write(b"gathering %d sample pulls within %d revisions\n" |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
508 |
% (count, len(actionrevs))) |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
509 |
if 1 < min_cache: |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
510 |
repo.ui.write(b" not caching ranges smaller than %d changesets\n" % min_cache) |
4737
46c990705a96
py3: replace xrange() by range()
Martin von Zweigbergk <martinvonz@google.com>
parents:
4208
diff
changeset
|
511 |
for i in range(count): |
5044
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
512 |
progress(repo.ui, b'gathering data', i, total=count) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
513 |
outgoing = takeonesample(repo, actionrevs) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
514 |
ranges = sliceoutgoing(repo, outgoing) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
515 |
hitranges = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
516 |
hitchanges = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
517 |
totalchanges = 0 |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
518 |
largeranges = [] |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
519 |
for rangeid, __ in ranges: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
520 |
length = rlen(rangeid) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
521 |
totalchanges += length |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
522 |
if bundlehits[rangeid]: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
523 |
hitranges += 1 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
524 |
hitchanges += rlen(rangeid) |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
525 |
if min_cache <= length: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
526 |
bundlehits[rangeid] += 1 |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
527 |
largeranges.append(rangeid) |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
528 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
529 |
stats = (len(outgoing.missing), |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
530 |
totalchanges, |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
531 |
hitchanges, |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
532 |
len(largeranges), |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
533 |
hitranges, |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
534 |
) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
535 |
pullstats.append(stats) |
5044
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
536 |
progress(repo.ui, b'gathering data', None) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
537 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
538 |
sizes = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
539 |
changesmissing = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
540 |
totalchanges = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
541 |
totalcached = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
542 |
changesratio = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
543 |
rangesratio = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
544 |
bundlecount = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
545 |
for entry in pullstats: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
546 |
sizes.append(entry[0]) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
547 |
changesmissing.append(entry[1] - entry[2]) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
548 |
changesratio.append(entry[2] / float(entry[1])) |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
549 |
if entry[3]: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
550 |
rangesratio.append(entry[4] / float(entry[3])) |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
551 |
else: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
552 |
rangesratio.append(1) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
553 |
bundlecount.append(entry[3]) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
554 |
totalchanges += entry[1] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
555 |
totalcached += entry[2] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
556 |
|
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
557 |
cachedsizes = [] |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
558 |
cachedhits = [] |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
559 |
for rangeid, hits in bundlehits.items(): |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
560 |
if hits <= 0: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
561 |
continue |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
562 |
length = rlen(rangeid) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
563 |
cachedsizes.append(length) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
564 |
cachedhits.append(hits) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
565 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
566 |
sizesdist = distribution(sizes) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
567 |
repo.ui.write(fmtdist(b'pull size', sizesdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
568 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
569 |
changesmissingdist = distribution(changesmissing) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
570 |
repo.ui.write(fmtdist(b'non-cached changesets', changesmissingdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
571 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
572 |
changesratiodist = distribution(changesratio) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
573 |
repo.ui.write(fmtdist(b'ratio of cached changesets', changesratiodist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
574 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
575 |
bundlecountdist = distribution(bundlecount) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
576 |
repo.ui.write(fmtdist(b'bundle count', bundlecountdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
577 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
578 |
rangesratiodist = distribution(rangesratio) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
579 |
repo.ui.write(fmtdist(b'ratio of cached bundles', rangesratiodist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
580 |
|
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
581 |
repo.ui.write(b'changesets served:\n') |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
582 |
repo.ui.write(b' total: %7d\n' % totalchanges) |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
583 |
repo.ui.write(b' from cache: %7d (%2d%%)\n' |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
584 |
% (totalcached, (totalcached * 100 // totalchanges))) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
585 |
repo.ui.write(b' bundle: %7d\n' % sum(bundlecount)) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
586 |
|
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
587 |
cachedsizesdist = distribution(cachedsizes) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
588 |
repo.ui.write(fmtdist(b'size of cached bundles', cachedsizesdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
589 |
|
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
590 |
cachedhitsdist = distribution(cachedhits) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
591 |
repo.ui.write(fmtdist(b'hit on cached bundles', cachedhitsdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
592 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
593 |
def takeonesample(repo, revs): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
594 |
node = repo.changelog.node |
5047
a8ac174010a5
pullbundle: random.sample() chokes on a spanset, turn it into a list instead
Anton Shestakov <av6@dwimlabs.net>
parents:
5046
diff
changeset
|
595 |
revs = list(revs) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
596 |
pulled = random.sample(revs, max(4, len(revs) // 1000)) |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
597 |
pulled = repo.revs(b'%ld::%ld', pulled, pulled) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
598 |
nodes = [node(r) for r in pulled] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
599 |
return outgoingfromnodes(repo, nodes) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
600 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
601 |
def distribution(data): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
602 |
data.sort() |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
603 |
length = len(data) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
604 |
return { |
4814
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
605 |
b'min': data[0], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
606 |
b'10%': data[length // 10], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
607 |
b'25%': data[length // 4], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
608 |
b'50%': data[length // 2], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
609 |
b'75%': data[(length // 4) * 3], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
610 |
b'90%': data[(length // 10) * 9], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
611 |
b'95%': data[(length // 20) * 19], |
48b30ff742cb
python3: use format-source to run byteify-strings in .py files
Raphaël Gomès <rgomes@octobus.net>
parents:
4807
diff
changeset
|
612 |
b'max': data[-1], |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
613 |
} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
614 |
|
5046
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
615 |
STATSFORMAT = b"""%(name)s: |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
616 |
min: %(min)r |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
617 |
10%%: %(10%)r |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
618 |
25%%: %(25%)r |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
619 |
50%%: %(50%)r |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
620 |
75%%: %(75%)r |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
621 |
90%%: %(90%)r |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
622 |
95%%: %(95%)r |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
623 |
max: %(max)r |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
624 |
""" |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
625 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
626 |
def fmtdist(name, data): |
5046
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
627 |
data[b'name'] = name |
ebd0f5c6098f
pullbundle: use % instead of .format() because % exists for bytes
Anton Shestakov <av6@dwimlabs.net>
parents:
5044
diff
changeset
|
628 |
return STATSFORMAT % data |
4929
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
629 |
|
5044
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
630 |
# hg <= 4.6 (bec1212eceaa) |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
631 |
if util.safehasattr(uimod.ui, 'makeprogress'): |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
632 |
def progress(ui, topic, pos, item=b"", unit=b"", total=None): |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
633 |
progress = ui.makeprogress(topic, unit, total) |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
634 |
if pos is not None: |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
635 |
progress.update(pos, item=item) |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
636 |
else: |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
637 |
progress.complete() |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
638 |
else: |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
639 |
def progress(ui, topic, pos, item=b"", unit=b"", total=None): |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
640 |
ui.progress(topic, pos, item, unit, total) |
b9179a034005
pullbundle: compatibility for progress in hg <= 4.6
Anton Shestakov <av6@dwimlabs.net>
parents:
5034
diff
changeset
|
641 |
|
4929
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
642 |
# nodemap.get and index.[has_node|rev|get_rev] |
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
643 |
# hg <= 5.3 (02802fa87b74) |
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
644 |
def getgetrev(cl): |
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
645 |
"""Returns index.get_rev or nodemap.get (for pre-5.3 Mercurial).""" |
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
646 |
if util.safehasattr(cl.index, 'get_rev'): |
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
647 |
return cl.index.get_rev |
bb2b4f6c99dc
compat: compatibility for cl.nodemap.get vs cl.index.get_rev
Anton Shestakov <av6@dwimlabs.net>
parents:
4814
diff
changeset
|
648 |
return cl.nodemap.get |