author | Pierre-Yves David <pierre-yves.david@octobus.net> |
Thu, 25 Oct 2018 13:04:08 +0200 | |
branch | mercurial-4.5 |
changeset 4205 | 6be09fb2c08c |
parent 4201 | 46cef25fbd96 |
child 4208 | 7d4c80c3f386 |
permissions | -rw-r--r-- |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
1 |
# Extension to provide automatic caching of bundle server for pull |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
2 |
# |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
3 |
# Copyright 2018 Pierre-Yves David <pierre-yves.david@ens-lyon.org> |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
4 |
# |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
5 |
# This software may be used and distributed according to the terms of the |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
6 |
# GNU General Public License version 2 or any later version. |
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
7 |
"""pullbundle: automatic server side bundle caching |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
8 |
|
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
9 |
General principle |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
10 |
================= |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
11 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
12 |
This extension provides a means for server to use pre-computed bundle for |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
13 |
serving arbitrary pulls. If missing, the necessary pre-computed bundle will be |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
14 |
generated on demand. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
15 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
16 |
To maximize usage of existing cached bundle, each pull will be served through |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
17 |
multiple bundles. The bundle will be created using "standard range" from the |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
18 |
"stablerange" principle. The "stablerange" concept if already used for |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
19 |
obsmarkers discovery in the evolve extensions. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
20 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
21 |
Using pull Bundle |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
22 |
================= |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
23 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
24 |
All configuration is only required server side. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
25 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
26 |
The "stablerange" code currently still live in the evolve extensions, so for |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
27 |
now enabling that extensions is required: |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
28 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
29 |
You need at minimum the following configuration: |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
30 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
31 |
[extensions] |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
32 |
evolve=yes |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
33 |
pullbundle=yes |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
34 |
[experimental] |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
35 |
obshashrange.warm-cache = yes |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
36 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
37 |
If you do not want to use evolution server side, you should disable obsmarkers exchange: |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
38 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
39 |
[experimental] |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
40 |
evolution.exchange=no |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
41 |
|
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
42 |
Extra Configuration |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
43 |
=================== |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
44 |
|
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
45 |
[pullbundle] |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
46 |
# By default bundles are stored `.hg/cache/pullbundles/. |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
47 |
# This can be changed with the following config: |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
48 |
cache-directory=/absolute/path |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
49 |
|
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
50 |
Implementation status |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
51 |
===================== |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
52 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
53 |
Both for stablerange and pullbundle use "simple" initial implementations. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
54 |
Theses implemenations focus on testing the algorithms and proving the features |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
55 |
works. Yet they are already useful and used in production. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
56 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
57 |
Performances are expected to greatly improved in the final implementation, |
4141
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
58 |
especially if some of it end up being compiled code. |
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
59 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
60 |
This first implementation lacks the ability to server the cached bundle from a |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
61 |
CDN. We'll want this limitation to be lifted quickly. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
62 |
|
4141
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
63 |
The way mercurial core report progress is designed for the receival of a single |
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
64 |
changegroup. So currently using pullbundle means flooding the user with output. |
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
65 |
This will have to be fixed. |
1421ff5c5c96
pullbundle: point out the output flooding issue from core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4140
diff
changeset
|
66 |
|
4132
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
67 |
Why is does this live in the same repository as evolve |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
68 |
====================================================== |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
69 |
|
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
70 |
There is no fundamental reasons for live in the same repository. However, the |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
71 |
stablerange data-structure lives in evolve, so it was simpler to put this new |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
72 |
extensions next to it. As soon as stable range have been upstreamed, we won't |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
73 |
need the dependency to the evolve extension anymore. |
afc933d32085
pullbundle: add extension documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4130
diff
changeset
|
74 |
""" |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
75 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
76 |
import collections |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
77 |
import errno |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
78 |
import random |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
79 |
import os |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
80 |
|
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
81 |
from mercurial import ( |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
82 |
changegroup, |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
83 |
discovery, |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
84 |
error, |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
85 |
exchange, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
86 |
narrowspec, |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
87 |
node as nodemod, |
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
88 |
registrar, |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
89 |
scmutil, |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
90 |
util, |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
91 |
) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
92 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
93 |
from mercurial.i18n import _ |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
94 |
|
4201
46cef25fbd96
pullbundle: drop erroneous flagging of the command as basic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4176
diff
changeset
|
95 |
__version__ = '0.1.1' |
4175
03c3206f44f3
pullbundle: document working version and update changelog to mention it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4149
diff
changeset
|
96 |
testedwith = '4.4 4.5 4.6 4.7.1' |
03c3206f44f3
pullbundle: document working version and update changelog to mention it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4149
diff
changeset
|
97 |
minimumhgversion = '4.4' |
4135
47f1d7b4305d
pullbundle: add basic extension metadata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4134
diff
changeset
|
98 |
buglink = 'https://bz.mercurial-scm.org/' |
47f1d7b4305d
pullbundle: add basic extension metadata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4134
diff
changeset
|
99 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
100 |
cmdtable = {} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
101 |
command = registrar.command(cmdtable) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
102 |
|
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
103 |
configtable = {} |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
104 |
configitem = registrar.configitem(configtable) |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
105 |
|
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
106 |
configitem('pullbundle', 'cache-directory', |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
107 |
default=None, |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
108 |
) |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
109 |
|
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
110 |
# generic wrapping |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
111 |
|
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
112 |
def uisetup(ui): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
113 |
exchange.getbundle2partsmapping['changegroup'] = _getbundlechangegrouppart |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
114 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
115 |
def _getbundlechangegrouppart(bundler, repo, source, bundlecaps=None, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
116 |
b2caps=None, heads=None, common=None, **kwargs): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
117 |
"""add a changegroup part to the requested bundle""" |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
118 |
if not kwargs.get(r'cg', True): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
119 |
return |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
120 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
121 |
version = '01' |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
122 |
cgversions = b2caps.get('changegroup') |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
123 |
if cgversions: # 3.1 and 3.2 ship with an empty value |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
124 |
cgversions = [v for v in cgversions |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
125 |
if v in changegroup.supportedoutgoingversions(repo)] |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
126 |
if not cgversions: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
127 |
raise ValueError(_('no common changegroup version')) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
128 |
version = max(cgversions) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
129 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
130 |
outgoing = exchange._computeoutgoing(repo, heads, common) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
131 |
if not outgoing.missing: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
132 |
return |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
133 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
134 |
if kwargs.get(r'narrow', False): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
135 |
include = sorted(filter(bool, kwargs.get(r'includepats', []))) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
136 |
exclude = sorted(filter(bool, kwargs.get(r'excludepats', []))) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
137 |
filematcher = narrowspec.match(repo.root, include=include, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
138 |
exclude=exclude) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
139 |
else: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
140 |
filematcher = None |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
141 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
142 |
# START OF ALTERED PART |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
143 |
makeallcgpart(bundler.newpart, repo, outgoing, version, source, bundlecaps, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
144 |
filematcher, cgversions) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
145 |
# END OF ALTERED PART |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
146 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
147 |
if kwargs.get(r'narrow', False) and (include or exclude): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
148 |
narrowspecpart = bundler.newpart('narrow:spec') |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
149 |
if include: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
150 |
narrowspecpart.addparam( |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
151 |
'include', '\n'.join(include), mandatory=True) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
152 |
if exclude: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
153 |
narrowspecpart.addparam( |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
154 |
'exclude', '\n'.join(exclude), mandatory=True) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
155 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
156 |
def makeallcgpart(newpart, repo, outgoing, version, source, |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
157 |
bundlecaps, filematcher, cgversions): |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
158 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
159 |
pullbundle = not filematcher |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
160 |
if pullbundle and not util.safehasattr(repo, 'stablerange'): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
161 |
repo.ui.warn('pullbundle: required extension "evolve" are missing, skipping pullbundle\n') |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
162 |
pullbundle = False |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
163 |
if filematcher: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
164 |
makeonecgpart(newpart, repo, None, outgoing, version, source, bundlecaps, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
165 |
filematcher, cgversions) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
166 |
else: |
4133
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
167 |
start = util.timer() |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
168 |
slices = sliceoutgoing(repo, outgoing) |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
169 |
end = util.timer() |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
170 |
msg = _('pullbundle-cache: "missing" set sliced into %d subranges ' |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
171 |
'in %s seconds\n') |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
172 |
repo.ui.write(msg % (len(slices), end - start)) |
1293625d274d
pullbundle: add an output about the time spent slicing the set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4132
diff
changeset
|
173 |
for sliceid, sliceout in slices: |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
174 |
makeonecgpart(newpart, repo, sliceid, sliceout, version, source, bundlecaps, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
175 |
filematcher, cgversions) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
176 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
177 |
# stable range slicing |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
178 |
|
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
179 |
DEBUG = False |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
180 |
|
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
181 |
def sliceoutgoing(repo, outgoing): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
182 |
cl = repo.changelog |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
183 |
rev = cl.nodemap.get |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
184 |
node = cl.node |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
185 |
revsort = repo.stablesort |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
186 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
187 |
missingrevs = set(rev(n) for n in outgoing.missing) |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
188 |
if DEBUG: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
189 |
ms = missingrevs.copy() |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
190 |
ss = [] |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
191 |
allslices = [] |
4147
88e922eca4e2
pullbundle: deal with head in a predicable order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4145
diff
changeset
|
192 |
missingheads = [rev(n) for n in sorted(outgoing.missingheads, reverse=True)] |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
193 |
for head in missingheads: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
194 |
localslices = [] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
195 |
localmissing = set(repo.revs('%ld and ::%d', missingrevs, head)) |
4149
4abec3a1208f
pullbundle: fix style warning
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4148
diff
changeset
|
196 |
thisrunmissing = localmissing.copy() |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
197 |
while localmissing: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
198 |
slicerevs = [] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
199 |
for r in revsort.walkfrom(repo, head): |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
200 |
if r not in thisrunmissing: |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
201 |
break |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
202 |
slicerevs.append(r) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
203 |
slicenodes = [node(r) for r in slicerevs] |
4140
9b71aa222f8e
pullbundle: issue local branches in the right order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4139
diff
changeset
|
204 |
localslices.append(canonicalslices(repo, slicenodes)) |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
205 |
if DEBUG: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
206 |
ss.append(slicerevs) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
207 |
missingrevs.difference_update(slicerevs) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
208 |
localmissing.difference_update(slicerevs) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
209 |
if localmissing: |
4148
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
210 |
heads = list(repo.revs('heads(%ld)', localmissing)) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
211 |
heads.sort(key=node) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
212 |
head = heads.pop() |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
213 |
if heads: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
214 |
thisrunmissing = repo.revs('%ld and only(%d, %ld)', |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
215 |
localmissing, |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
216 |
head, |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
217 |
heads) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
218 |
else: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
219 |
thisrunmissing = localmissing.copy() |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
220 |
if DEBUG: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
221 |
for s in reversed(ss): |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
222 |
ms -= set(s) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
223 |
missingbase = repo.revs('parents(%ld) and %ld', s, ms) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
224 |
if missingbase: |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
225 |
repo.ui.write_err('!!! rev bundled while parents missing\n') |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
226 |
repo.ui.write_err(' parent: %s\n' % list(missingbase)) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
227 |
pb = repo.revs('%ld and children(%ld)', s, missingbase) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
228 |
repo.ui.write_err(' children: %s\n' % list(pb)) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
229 |
h = repo.revs('heads(%ld)', s) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
230 |
repo.ui.write_err(' heads: %s\n' % list(h)) |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
231 |
raise error.ProgrammingError('issuing a range before its parents') |
5ef93dbace5e
pullbundle: improves management of multiple branch while dealing with a head
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4147
diff
changeset
|
232 |
|
4140
9b71aa222f8e
pullbundle: issue local branches in the right order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4139
diff
changeset
|
233 |
for s in reversed(localslices): |
9b71aa222f8e
pullbundle: issue local branches in the right order
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4139
diff
changeset
|
234 |
allslices.extend(s) |
4136
be3a94d3105f
pullbundle: attempt to save stablerange cache after each computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4135
diff
changeset
|
235 |
# unknown subrange might had to be computed |
be3a94d3105f
pullbundle: attempt to save stablerange cache after each computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4135
diff
changeset
|
236 |
repo.stablerange.save(repo) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
237 |
return [(rangeid, outgoingfromnodes(repo, nodes)) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
238 |
for rangeid, nodes in allslices] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
239 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
240 |
def canonicalslices(repo, nodes): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
241 |
depth = repo.depthcache.get |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
242 |
stablerange = repo.stablerange |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
243 |
rangelength = lambda x: stablerange.rangelength(repo, x) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
244 |
headrev = repo.changelog.rev(nodes[0]) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
245 |
nbrevs = len(nodes) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
246 |
headdepth = depth(headrev) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
247 |
skipped = headdepth - nbrevs |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
248 |
rangeid = (headrev, skipped) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
249 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
250 |
subranges = canonicalsubranges(repo, stablerange, rangeid) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
251 |
idx = 0 |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
252 |
slices = [] |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
253 |
nodes.reverse() |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
254 |
for rangeid in subranges: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
255 |
size = rangelength(rangeid) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
256 |
slices.append((rangeid, nodes[idx:idx + size])) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
257 |
idx += size |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
258 |
### slow code block to validate ranges content |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
259 |
# rev = repo.changelog.nodemap.get |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
260 |
# for ri, ns in slices: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
261 |
# a = set(rev(n) for n in ns) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
262 |
# b = set(repo.stablerange.revsfromrange(repo, ri)) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
263 |
# l = repo.stablerange.rangelength(repo, ri) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
264 |
# repo.ui.write('range-length: %d-%d %s %s\n' % (ri[0], ri[1], l, len(a))) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
265 |
# if a != b: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
266 |
# d = (ri[0], ri[1], b - a, a - b) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
267 |
# repo.ui.write("mismatching content: %d-%d -%s +%s\n" % d) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
268 |
return slices |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
269 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
270 |
def canonicalsubranges(repo, stablerange, rangeid): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
271 |
"""slice a size of nodes into most reusable subranges |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
272 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
273 |
We try to slice a range into a set of "largest" and "canonical" stable |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
274 |
range. |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
275 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
276 |
It might make sense to move this function as a 'stablerange' method. |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
277 |
""" |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
278 |
headrev, skip = rangeid |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
279 |
rangedepth = stablerange.depthrev(repo, rangeid[0]) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
280 |
canonicals = [] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
281 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
282 |
# 0. find the first power of 2 higher than this range depth |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
283 |
cursor = 1 |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
284 |
while cursor <= rangedepth: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
285 |
cursor *= 2 |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
286 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
287 |
# 1. find first cupt |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
288 |
precut = cut = 0 |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
289 |
while True: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
290 |
if skip <= cut: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
291 |
break |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
292 |
if cut + cursor < rangedepth: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
293 |
precut = cut |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
294 |
cut += cursor |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
295 |
if cursor == 1: |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
296 |
break |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
297 |
cursor //= 2 |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
298 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
299 |
# 2. optimise, bottom part |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
300 |
if skip != cut: |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
301 |
currentsize = tailsize = cut - skip |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
302 |
assert 0 < tailsize, tailsize |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
303 |
|
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
304 |
# we need to take several "standard cut" in the bottom part |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
305 |
# |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
306 |
# This is similar to what we will do for the top part, we reusing the |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
307 |
# existing structure is a bit more complex. |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
308 |
allcuts = list(reversed(standardcut(tailsize))) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
309 |
prerange = (headrev, precut) |
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
310 |
### slow code block to check we operate on the right data |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
311 |
# rev = repo.changelog.nodemap.get |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
312 |
# allrevs = [rev(n) for n in nodes] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
313 |
# allrevs.reverse() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
314 |
# prerevs = repo.stablerange.revsfromrange(repo, prerange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
315 |
# assert allrevs == prerevs[(len(prerevs) - len(allrevs)):] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
316 |
# end of check |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
317 |
sub = list(stablerange.subranges(repo, prerange)[:-1]) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
318 |
|
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
319 |
bottomranges = [] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
320 |
# XXX we might be able to reuse core stable-range logic instead of |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
321 |
# redoing this manually |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
322 |
currentrange = sub.pop() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
323 |
currentsize = stablerange.rangelength(repo, currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
324 |
currentcut = None |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
325 |
while allcuts or currentcut is not None: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
326 |
# get the next cut if needed |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
327 |
if currentcut is None: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
328 |
currentcut = allcuts.pop() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
329 |
# deal attemp a cut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
330 |
if currentsize == currentcut: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
331 |
bottomranges.append(currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
332 |
currentcut = None |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
333 |
elif currentsize < currentcut: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
334 |
bottomranges.append(currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
335 |
currentcut -= currentsize |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
336 |
else: # currentsize > currentcut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
337 |
newskip = currentrange[1] + (currentsize - currentcut) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
338 |
currentsub = stablerange._slicesrangeat(repo, currentrange, newskip) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
339 |
bottomranges.append(currentsub.pop()) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
340 |
sub.extend(currentsub) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
341 |
currentcut = None |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
342 |
currentrange = sub.pop() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
343 |
currentsize = stablerange.rangelength(repo, currentrange) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
344 |
bottomranges.reverse() |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
345 |
canonicals.extend(bottomranges) |
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
346 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
347 |
# 3. take recursive subrange until we get to a power of two size? |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
348 |
current = (headrev, cut) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
349 |
while not poweroftwo(stablerange.rangelength(repo, current)): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
350 |
sub = stablerange.subranges(repo, current) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
351 |
canonicals.extend(sub[:-1]) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
352 |
current = sub[-1] |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
353 |
canonicals.append(current) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
354 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
355 |
return canonicals |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
356 |
|
4138
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
357 |
def standardcut(size): |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
358 |
assert 0 < size |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
359 |
# 0. find the first power of 2 higher than this range depth |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
360 |
cut = 1 |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
361 |
while cut <= size: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
362 |
cut *= 2 |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
363 |
|
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
364 |
allcuts = [] |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
365 |
# 1. find all standard expected cut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
366 |
while 1 < cut and size: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
367 |
cut //= 2 |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
368 |
if cut <= size: |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
369 |
allcuts.append(cut) |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
370 |
size -= cut |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
371 |
return allcuts |
cfdc6f55599b
pullbundle: improve slicing of the lower part of range
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4136
diff
changeset
|
372 |
|
4129
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
373 |
def poweroftwo(num): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
374 |
return num and not num & (num - 1) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
375 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
376 |
def outgoingfromnodes(repo, nodes): |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
377 |
return discovery.outgoing(repo, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
378 |
missingroots=nodes, |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
379 |
missingheads=nodes) |
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
380 |
|
bc4e62a1cb82
pullbundle: slice pull into multiple ranges
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4128
diff
changeset
|
381 |
# changegroup part construction |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
382 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
383 |
def _changegroupinfo(repo, nodes, source): |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
384 |
if repo.ui.verbose or source == 'bundle': |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
385 |
repo.ui.status(_("%d changesets found\n") % len(nodes)) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
386 |
|
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
387 |
def _makenewstream(newpart, repo, outgoing, version, source, |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
388 |
bundlecaps, filematcher, cgversions): |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
389 |
old = changegroup._changegroupinfo |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
390 |
try: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
391 |
changegroup._changegroupinfo = _changegroupinfo |
4145
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
392 |
if filematcher is not None: |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
393 |
cgstream = changegroup.makestream(repo, outgoing, version, source, |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
394 |
bundlecaps=bundlecaps, |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
395 |
filematcher=filematcher) |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
396 |
else: |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
397 |
cgstream = changegroup.makestream(repo, outgoing, version, source, |
08b3c370e8b3
pullbundle: compat with Mercurial 4.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4144
diff
changeset
|
398 |
bundlecaps=bundlecaps) |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
399 |
finally: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
400 |
changegroup._changegroupinfo = old |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
401 |
|
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
402 |
nbchanges = len(outgoing.missing) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
403 |
pversion = None |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
404 |
if cgversions: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
405 |
pversion = version |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
406 |
return (cgstream, nbchanges, pversion) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
407 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
408 |
def _makepartfromstream(newpart, repo, cgstream, nbchanges, version): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
409 |
# same as upstream code |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
410 |
|
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
411 |
part = newpart('changegroup', data=cgstream) |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
412 |
if version: |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
413 |
part.addparam('version', version) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
414 |
|
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
415 |
part.addparam('nbchanges', '%d' % nbchanges, |
4128
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
416 |
mandatory=False) |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
417 |
|
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
418 |
if 'treemanifest' in repo.requirements: |
4e5ec9ae682e
pullbundle: initial commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
419 |
part.addparam('treemanifest', '1') |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
420 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
421 |
# cache management |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
422 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
423 |
def cachedir(repo): |
4134
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
424 |
cachedir = repo.ui.config('pullbundle', 'cache-directory') |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
425 |
if cachedir is not None: |
ab77f37fedf3
pullbundle: add a config option for the cache directory
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4133
diff
changeset
|
426 |
return cachedir |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
427 |
return repo.cachevfs.join('pullbundles') |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
428 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
429 |
def getcache(repo, bundlename): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
430 |
cdir = cachedir(repo) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
431 |
bundlepath = os.path.join(cdir, bundlename) |
4139
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
432 |
if not os.path.exists(bundlepath): |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
433 |
return None |
4139
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
434 |
# delay file opening as much as possible this introduce a small race |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
435 |
# condition if someone remove the file before we actually use it. However |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
436 |
# opening too many file will not work. |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
437 |
|
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
438 |
def data(): |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
439 |
with open(bundlepath, 'rb') as fd: |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
440 |
for chunk in util.filechunkiter(fd): |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
441 |
yield chunk |
2bd652bece97
pullbundle: delay cache file opening
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4138
diff
changeset
|
442 |
return data() |
4130
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
443 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
444 |
def cachewriter(repo, bundlename, stream): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
445 |
cdir = cachedir(repo) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
446 |
bundlepath = os.path.join(cdir, bundlename) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
447 |
try: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
448 |
os.makedirs(cdir) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
449 |
except OSError as exc: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
450 |
if exc.errno == errno.EEXIST: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
451 |
pass |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
452 |
with util.atomictempfile(bundlepath) as cachefile: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
453 |
for chunk in stream: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
454 |
cachefile.write(chunk) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
455 |
yield chunk |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
456 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
457 |
BUNDLEMASK = "%s-%s-%010iskip-%010isize.hg" |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
458 |
|
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
459 |
def makeonecgpart(newpart, repo, rangeid, outgoing, version, source, |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
460 |
bundlecaps, filematcher, cgversions): |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
461 |
bundlename = cachedata = None |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
462 |
if rangeid is not None: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
463 |
nbchanges = repo.stablerange.rangelength(repo, rangeid) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
464 |
headnode = nodemod.hex(repo.changelog.node(rangeid[0])) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
465 |
# XXX do we need to use cgversion in there? |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
466 |
bundlename = BUNDLEMASK % (version, headnode, rangeid[1], nbchanges) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
467 |
cachedata = getcache(repo, bundlename) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
468 |
if cachedata is None: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
469 |
partdata = _makenewstream(newpart, repo, outgoing, version, source, |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
470 |
bundlecaps, filematcher, cgversions) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
471 |
if bundlename is not None: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
472 |
cgstream = cachewriter(repo, bundlename, partdata[0]) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
473 |
partdata = (cgstream,) + partdata[1:] |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
474 |
else: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
475 |
if repo.ui.verbose or source == 'bundle': |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
476 |
repo.ui.status(_("%d changesets found in caches\n") % nbchanges) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
477 |
pversion = None |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
478 |
if cgversions: |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
479 |
pversion = version |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
480 |
partdata = (cachedata, nbchanges, pversion) |
a1f6b8211016
pullbundle: add caching logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4129
diff
changeset
|
481 |
return _makepartfromstream(newpart, repo, *partdata) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
482 |
|
4201
46cef25fbd96
pullbundle: drop erroneous flagging of the command as basic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4176
diff
changeset
|
483 |
@command('debugpullbundlecacheoverlap', |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
484 |
[('', 'count', 100, _('of "client" pulling')), |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
485 |
('', 'min-cache', 1, _('minimum size of cached bundle')), |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
486 |
], |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
487 |
_('hg debugpullbundlecacheoverlap [--client 100] REVSET')) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
488 |
def debugpullbundlecacheoverlap(ui, repo, *revs, **opts): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
489 |
'''Display statistic on bundle cache hit |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
490 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
491 |
This command "simulate pulls from multiple clients. Each using a random |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
492 |
subset of revisions defined by REVSET. And display statistic about the |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
493 |
overlap in bundle necessary to serve them. |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
494 |
''' |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
495 |
actionrevs = scmutil.revrange(repo, revs) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
496 |
if not revs: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
497 |
raise error.Abort('No revision selected') |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
498 |
count = opts['count'] |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
499 |
min_cache = opts['min_cache'] |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
500 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
501 |
bundlehits = collections.defaultdict(lambda: 0) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
502 |
pullstats = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
503 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
504 |
rlen = lambda rangeid: repo.stablerange.rangelength(repo, rangeid) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
505 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
506 |
repo.ui.write("gathering %d sample pulls within %d revisions\n" |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
507 |
% (count, len(actionrevs))) |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
508 |
if 1 < min_cache: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
509 |
repo.ui.write(" not caching ranges smaller than %d changesets\n" % min_cache) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
510 |
for i in xrange(count): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
511 |
repo.ui.progress('gathering data', i, total=count) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
512 |
outgoing = takeonesample(repo, actionrevs) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
513 |
ranges = sliceoutgoing(repo, outgoing) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
514 |
hitranges = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
515 |
hitchanges = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
516 |
totalchanges = 0 |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
517 |
largeranges = [] |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
518 |
for rangeid, __ in ranges: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
519 |
length = rlen(rangeid) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
520 |
totalchanges += length |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
521 |
if bundlehits[rangeid]: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
522 |
hitranges += 1 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
523 |
hitchanges += rlen(rangeid) |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
524 |
if min_cache <= length: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
525 |
bundlehits[rangeid] += 1 |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
526 |
largeranges.append(rangeid) |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
527 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
528 |
stats = (len(outgoing.missing), |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
529 |
totalchanges, |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
530 |
hitchanges, |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
531 |
len(largeranges), |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
532 |
hitranges, |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
533 |
) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
534 |
pullstats.append(stats) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
535 |
repo.ui.progress('gathering data', None) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
536 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
537 |
sizes = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
538 |
changesmissing = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
539 |
totalchanges = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
540 |
totalcached = 0 |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
541 |
changesratio = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
542 |
rangesratio = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
543 |
bundlecount = [] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
544 |
for entry in pullstats: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
545 |
sizes.append(entry[0]) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
546 |
changesmissing.append(entry[1] - entry[2]) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
547 |
changesratio.append(entry[2] / float(entry[1])) |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
548 |
if entry[3]: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
549 |
rangesratio.append(entry[4] / float(entry[3])) |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
550 |
else: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
551 |
rangesratio.append(1) |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
552 |
bundlecount.append(entry[3]) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
553 |
totalchanges += entry[1] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
554 |
totalcached += entry[2] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
555 |
|
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
556 |
cachedsizes = [] |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
557 |
cachedhits = [] |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
558 |
for rangeid, hits in bundlehits.items(): |
4144
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
559 |
if hits <= 0: |
b5cd26712e4b
pullbundle: add some information on the cached change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4143
diff
changeset
|
560 |
continue |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
561 |
length = rlen(rangeid) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
562 |
cachedsizes.append(length) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
563 |
cachedhits.append(hits) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
564 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
565 |
sizesdist = distribution(sizes) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
566 |
repo.ui.write(fmtdist('pull size', sizesdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
567 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
568 |
changesmissingdist = distribution(changesmissing) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
569 |
repo.ui.write(fmtdist('non-cached changesets', changesmissingdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
570 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
571 |
changesratiodist = distribution(changesratio) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
572 |
repo.ui.write(fmtdist('ratio of cached changesets', changesratiodist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
573 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
574 |
bundlecountdist = distribution(bundlecount) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
575 |
repo.ui.write(fmtdist('bundle count', bundlecountdist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
576 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
577 |
rangesratiodist = distribution(rangesratio) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
578 |
repo.ui.write(fmtdist('ratio of cached bundles', rangesratiodist)) |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
579 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
580 |
repo.ui.write('changesets served:\n') |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
581 |
repo.ui.write(' total: %7d\n' % totalchanges) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
582 |
repo.ui.write(' from cache: %7d (%2d%%)\n' |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
583 |
% (totalcached, (totalcached * 100 // totalchanges))) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
584 |
repo.ui.write(' bundle: %7d\n' % sum(bundlecount)) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
585 |
|
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
586 |
cachedsizesdist = distribution(cachedsizes) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
587 |
repo.ui.write(fmtdist('size of cached bundles', cachedsizesdist)) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
588 |
|
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
589 |
cachedhitsdist = distribution(cachedhits) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
590 |
repo.ui.write(fmtdist('hit on cached bundles', cachedhitsdist)) |
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
591 |
|
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
592 |
def takeonesample(repo, revs): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
593 |
node = repo.changelog.node |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
594 |
pulled = random.sample(revs, max(4, len(revs) // 1000)) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
595 |
pulled = repo.revs('%ld::%ld', pulled, pulled) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
596 |
nodes = [node(r) for r in pulled] |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
597 |
return outgoingfromnodes(repo, nodes) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
598 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
599 |
def distribution(data): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
600 |
data.sort() |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
601 |
length = len(data) |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
602 |
return { |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
603 |
'min': data[0], |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
604 |
'10%': data[length // 10], |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
605 |
'25%': data[length // 4], |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
606 |
'50%': data[length // 2], |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
607 |
'75%': data[(length // 4) * 3], |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
608 |
'90%': data[(length // 10) * 9], |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
609 |
'95%': data[(length // 20) * 19], |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
610 |
'max': data[-1], |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
611 |
} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
612 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
613 |
STATSFORMAT = """{name}: |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
614 |
min: {min} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
615 |
10%: {10%} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
616 |
25%: {25%} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
617 |
50%: {50%} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
618 |
75%: {75%} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
619 |
90%: {90%} |
4143
2ddd8785f8d7
pullbundle: add data about the cached bundle size and hit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4142
diff
changeset
|
620 |
95%: {95%} |
4142
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
621 |
max: {max} |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
622 |
""" |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
623 |
|
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
624 |
def fmtdist(name, data): |
683ceec8d37e
pullbundle: add a command to generate cache hit statistic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
4141
diff
changeset
|
625 |
return STATSFORMAT.format(name=name, **data) |