1 .. _TutosPhotoWebSiteSecurity: |
|
2 |
|
3 Security, testing and migration |
|
4 ------------------------------- |
|
5 |
|
6 This part will cover various topics: |
|
7 |
|
8 * configuring security |
|
9 * migrating existing instance |
|
10 * writing some unit tests |
|
11 |
|
12 Here is the ``read`` security model I want: |
|
13 |
|
14 * folders, files, images and comments should have one of the following visibility: |
|
15 |
|
16 - ``public``, everyone can see it |
|
17 - ``authenticated``, only authenticated users can see it |
|
18 - ``restricted``, only a subset of authenticated users can see it |
|
19 |
|
20 * managers (e.g. me) can see everything |
|
21 * only authenticated users can see people |
|
22 * everyone can see classifier entities, such as tag and zone |
|
23 |
|
24 Also, unless explicitly specified, the visibility of an image should be the same as |
|
25 its parent folder, as well as visibility of a comment should be the same as the |
|
26 commented entity. If there is no parent entity, the default visibility is |
|
27 ``authenticated``. |
|
28 |
|
29 Regarding write security, that's much easier: |
|
30 * anonymous can't write anything |
|
31 * authenticated users can only add comment |
|
32 * managers will add the remaining stuff |
|
33 |
|
34 Now, let's implement that! |
|
35 |
|
36 Proper security in CubicWeb is done at the schema level, so you don't have to |
|
37 bother with it in views: users will only see what they can see automatically. |
|
38 |
|
39 .. _adv_tuto_security: |
|
40 |
|
41 Step 1: configuring security into the schema |
|
42 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
43 |
|
44 In schema, you can grant access according to groups, or to some RQL expressions: |
|
45 users get access if the expression returns some results. To implement the read |
|
46 security defined earlier, groups are not enough, we'll need some RQL expression. Here |
|
47 is the idea: |
|
48 |
|
49 * add a `visibility` attribute on Folder, File and Comment, which may be one of |
|
50 the value explained above |
|
51 |
|
52 * add a `may_be_read_by` relation from Folder, File and Comment to users, |
|
53 which will define who can see the entity |
|
54 |
|
55 * security propagation will be done in hook. |
|
56 |
|
57 So the first thing to do is to modify my cube's schema.py to define those |
|
58 relations: |
|
59 |
|
60 .. sourcecode:: python |
|
61 |
|
62 from yams.constraints import StaticVocabularyConstraint |
|
63 |
|
64 class visibility(RelationDefinition): |
|
65 subject = ('Folder', 'File', 'Comment') |
|
66 object = 'String' |
|
67 constraints = [StaticVocabularyConstraint(('public', 'authenticated', |
|
68 'restricted', 'parent'))] |
|
69 default = 'parent' |
|
70 cardinality = '11' # required |
|
71 |
|
72 class may_be_read_by(RelationDefinition): |
|
73 __permissions__ = { |
|
74 'read': ('managers', 'users'), |
|
75 'add': ('managers',), |
|
76 'delete': ('managers',), |
|
77 } |
|
78 |
|
79 subject = ('Folder', 'File', 'Comment',) |
|
80 object = 'CWUser' |
|
81 |
|
82 We can note the following points: |
|
83 |
|
84 * we've added a new `visibility` attribute to folder, file, image and comment |
|
85 using a `RelationDefinition` |
|
86 |
|
87 * `cardinality = '11'` means this attribute is required. This is usually hidden |
|
88 under the `required` argument given to the `String` constructor, but we can |
|
89 rely on this here (same thing for StaticVocabularyConstraint, which is usually |
|
90 hidden by the `vocabulary` argument) |
|
91 |
|
92 * the `parent` possible value will be used for visibility propagation |
|
93 |
|
94 * think to secure the `may_be_read_by` permissions, else any user can add/delete it |
|
95 by default, which somewhat breaks our security model... |
|
96 |
|
97 Now, we should be able to define security rules in the schema, based on these new |
|
98 attribute and relation. Here is the code to add to *schema.py*: |
|
99 |
|
100 .. sourcecode:: python |
|
101 |
|
102 from cubicweb.schema import ERQLExpression |
|
103 |
|
104 VISIBILITY_PERMISSIONS = { |
|
105 'read': ('managers', |
|
106 ERQLExpression('X visibility "public"'), |
|
107 ERQLExpression('X may_be_read_by U')), |
|
108 'add': ('managers',), |
|
109 'update': ('managers', 'owners',), |
|
110 'delete': ('managers', 'owners'), |
|
111 } |
|
112 AUTH_ONLY_PERMISSIONS = { |
|
113 'read': ('managers', 'users'), |
|
114 'add': ('managers',), |
|
115 'update': ('managers', 'owners',), |
|
116 'delete': ('managers', 'owners'), |
|
117 } |
|
118 CLASSIFIERS_PERMISSIONS = { |
|
119 'read': ('managers', 'users', 'guests'), |
|
120 'add': ('managers',), |
|
121 'update': ('managers', 'owners',), |
|
122 'delete': ('managers', 'owners'), |
|
123 } |
|
124 |
|
125 from cubes.folder.schema import Folder |
|
126 from cubes.file.schema import File |
|
127 from cubes.comment.schema import Comment |
|
128 from cubes.person.schema import Person |
|
129 from cubes.zone.schema import Zone |
|
130 from cubes.tag.schema import Tag |
|
131 |
|
132 Folder.__permissions__ = VISIBILITY_PERMISSIONS |
|
133 File.__permissions__ = VISIBILITY_PERMISSIONS |
|
134 Comment.__permissions__ = VISIBILITY_PERMISSIONS.copy() |
|
135 Comment.__permissions__['add'] = ('managers', 'users',) |
|
136 Person.__permissions__ = AUTH_ONLY_PERMISSIONS |
|
137 Zone.__permissions__ = CLASSIFIERS_PERMISSIONS |
|
138 Tag.__permissions__ = CLASSIFIERS_PERMISSIONS |
|
139 |
|
140 What's important in there: |
|
141 |
|
142 * `VISIBILITY_PERMISSIONS` provides read access to managers group, if |
|
143 `visibility` attribute's value is 'public', or if user (designed by the 'U' |
|
144 variable in the expression) is linked to the entity (the 'X' variable) through |
|
145 the `may_be_read_by` permission |
|
146 |
|
147 * we modify permissions of the entity types we use by importing them and |
|
148 modifying their `__permissions__` attribute |
|
149 |
|
150 * notice the `.copy()`: we only want to modify 'add' permission for `Comment`, |
|
151 not for all entity types using `VISIBILITY_PERMISSIONS`! |
|
152 |
|
153 * the remaining part of the security model is done using regular groups: |
|
154 |
|
155 - `users` is the group to which all authenticated users will belong |
|
156 - `guests` is the group of anonymous users |
|
157 |
|
158 |
|
159 .. _adv_tuto_security_propagation: |
|
160 |
|
161 Step 2: security propagation in hooks |
|
162 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
163 |
|
164 To fullfill the requirements, we have to implement:: |
|
165 |
|
166 Also, unless explicity specified, visibility of an image should be the same as |
|
167 its parent folder, as well as visibility of a comment should be the same as the |
|
168 commented entity. |
|
169 |
|
170 This kind of `active` rule will be done using CubicWeb's hook |
|
171 system. Hooks are triggered on database events such as addition of a new |
|
172 entity or relation. |
|
173 |
|
174 The tricky part of the requirement is in *unless explicitly specified*, notably |
|
175 because when the entity is added, we don't know yet its 'parent' |
|
176 entity (e.g. Folder of an File, File commented by a Comment). To handle such things, |
|
177 CubicWeb provides `Operation`, which allow to schedule things to do at commit time. |
|
178 |
|
179 In our case we will: |
|
180 |
|
181 * on entity creation, schedule an operation that will set default visibility |
|
182 |
|
183 * when a "parent" relation is added, propagate parent's visibility unless the |
|
184 child already has a visibility set |
|
185 |
|
186 Here is the code in cube's *hooks.py*: |
|
187 |
|
188 .. sourcecode:: python |
|
189 |
|
190 from cubicweb.predicates import is_instance |
|
191 from cubicweb.server import hook |
|
192 |
|
193 class SetVisibilityOp(hook.DataOperationMixIn, hook.Operation): |
|
194 |
|
195 def precommit_event(self): |
|
196 for eid in self.get_data(): |
|
197 entity = self.session.entity_from_eid(eid) |
|
198 if entity.visibility == 'parent': |
|
199 entity.cw_set(visibility=u'authenticated') |
|
200 |
|
201 class SetVisibilityHook(hook.Hook): |
|
202 __regid__ = 'sytweb.setvisibility' |
|
203 __select__ = hook.Hook.__select__ & is_instance('Folder', 'File', 'Comment') |
|
204 events = ('after_add_entity',) |
|
205 |
|
206 def __call__(self): |
|
207 SetVisibilityOp.get_instance(self._cw).add_data(self.entity.eid) |
|
208 |
|
209 class SetParentVisibilityHook(hook.Hook): |
|
210 __regid__ = 'sytweb.setparentvisibility' |
|
211 __select__ = hook.Hook.__select__ & hook.match_rtype('filed_under', 'comments') |
|
212 events = ('after_add_relation',) |
|
213 |
|
214 def __call__(self): |
|
215 parent = self._cw.entity_from_eid(self.eidto) |
|
216 child = self._cw.entity_from_eid(self.eidfrom) |
|
217 if child.visibility == 'parent': |
|
218 child.cw_set(visibility=parent.visibility) |
|
219 |
|
220 Notice: |
|
221 |
|
222 * hooks are application objects, hence have selectors that should match entity or |
|
223 relation types to which the hook applies. To match a relation type, we use the |
|
224 hook specific `match_rtype` selector. |
|
225 |
|
226 * usage of `DataOperationMixIn`: instead of adding an operation for each added entity, |
|
227 DataOperationMixIn allows to create a single one and to store entity's eids to be |
|
228 processed in the transaction data. This is a good pratice to avoid heavy |
|
229 operations manipulation cost when creating a lot of entities in the same |
|
230 transaction. |
|
231 |
|
232 * the `precommit_event` method of the operation will be called at transaction's |
|
233 commit time. |
|
234 |
|
235 * in a hook, `self._cw` is the repository session, not a web request as usually |
|
236 in views |
|
237 |
|
238 * according to hook's event, you have access to different attributes on the hook |
|
239 instance. Here: |
|
240 |
|
241 - `self.entity` is the newly added entity on 'after_add_entity' events |
|
242 |
|
243 - `self.eidfrom` / `self.eidto` are the eid of the subject / object entity on |
|
244 'after_add_relation' events (you may also get the relation type using |
|
245 `self.rtype`) |
|
246 |
|
247 The `parent` visibility value is used to tell "propagate using parent security" |
|
248 because we want that attribute to be required, so we can't use None value else |
|
249 we'll get an error before we get any chance to propagate... |
|
250 |
|
251 Now, we also want to propagate the `may_be_read_by` relation. Fortunately, |
|
252 CubicWeb provides some base hook classes for such things, so we only have to add |
|
253 the following code to *hooks.py*: |
|
254 |
|
255 .. sourcecode:: python |
|
256 |
|
257 # relations where the "parent" entity is the subject |
|
258 S_RELS = set() |
|
259 # relations where the "parent" entity is the object |
|
260 O_RELS = set(('filed_under', 'comments',)) |
|
261 |
|
262 class AddEntitySecurityPropagationHook(hook.PropagateRelationHook): |
|
263 """propagate permissions when new entity are added""" |
|
264 __regid__ = 'sytweb.addentity_security_propagation' |
|
265 __select__ = (hook.PropagateRelationHook.__select__ |
|
266 & hook.match_rtype_sets(S_RELS, O_RELS)) |
|
267 main_rtype = 'may_be_read_by' |
|
268 subject_relations = S_RELS |
|
269 object_relations = O_RELS |
|
270 |
|
271 class AddPermissionSecurityPropagationHook(hook.PropagateRelationAddHook): |
|
272 """propagate permissions when new entity are added""" |
|
273 __regid__ = 'sytweb.addperm_security_propagation' |
|
274 __select__ = (hook.PropagateRelationAddHook.__select__ |
|
275 & hook.match_rtype('may_be_read_by',)) |
|
276 subject_relations = S_RELS |
|
277 object_relations = O_RELS |
|
278 |
|
279 class DelPermissionSecurityPropagationHook(hook.PropagateRelationDelHook): |
|
280 __regid__ = 'sytweb.delperm_security_propagation' |
|
281 __select__ = (hook.PropagateRelationDelHook.__select__ |
|
282 & hook.match_rtype('may_be_read_by',)) |
|
283 subject_relations = S_RELS |
|
284 object_relations = O_RELS |
|
285 |
|
286 * the `AddEntitySecurityPropagationHook` will propagate the relation |
|
287 when `filed_under` or `comments` relations are added |
|
288 |
|
289 - the `S_RELS` and `O_RELS` set as well as the `match_rtype_sets` selector are |
|
290 used here so that if my cube is used by another one, it'll be able to |
|
291 configure security propagation by simply adding relation to one of the two |
|
292 sets. |
|
293 |
|
294 * the two others will propagate permissions changes on parent entities to |
|
295 children entities |
|
296 |
|
297 |
|
298 .. _adv_tuto_tesing_security: |
|
299 |
|
300 Step 3: testing our security |
|
301 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
302 |
|
303 Security is tricky. Writing some tests for it is a very good idea. You should |
|
304 even write them first, as Test Driven Development recommends! |
|
305 |
|
306 Here is a small test case that will check the basis of our security |
|
307 model, in *test/unittest_sytweb.py*: |
|
308 |
|
309 .. sourcecode:: python |
|
310 |
|
311 from cubicweb.devtools.testlib import CubicWebTC |
|
312 from cubicweb import Binary |
|
313 |
|
314 class SecurityTC(CubicWebTC): |
|
315 |
|
316 def test_visibility_propagation(self): |
|
317 with self.admin_access.repo_cnx() as cnx: |
|
318 # create a user for later security checks |
|
319 toto = self.create_user(cnx, 'toto') |
|
320 cnx.commit() |
|
321 # init some data using the default manager connection |
|
322 folder = cnx.create_entity('Folder', |
|
323 name=u'restricted', |
|
324 visibility=u'restricted') |
|
325 photo1 = cnx.create_entity('File', |
|
326 data_name=u'photo1.jpg', |
|
327 data=Binary('xxx'), |
|
328 filed_under=folder) |
|
329 cnx.commit() |
|
330 # visibility propagation |
|
331 self.assertEquals(photo1.visibility, 'restricted') |
|
332 # unless explicitly specified |
|
333 photo2 = cnx.create_entity('File', |
|
334 data_name=u'photo2.jpg', |
|
335 data=Binary('xxx'), |
|
336 visibility=u'public', |
|
337 filed_under=folder) |
|
338 cnx.commit() |
|
339 self.assertEquals(photo2.visibility, 'public') |
|
340 with self.new_access('toto').repo_cnx() as cnx: |
|
341 # test security |
|
342 self.assertEqual(1, len(cnx.execute('File X'))) # only the public one |
|
343 self.assertEqual(0, len(cnx.execute('Folder X'))) # restricted... |
|
344 with self.admin_access.repo_cnx() as cnx: |
|
345 # may_be_read_by propagation |
|
346 folder = cnx.entity_from_eid(folder.eid) |
|
347 folder.cw_set(may_be_read_by=toto) |
|
348 cnx.commit() |
|
349 with self.new_access('toto').repo_cnx() as cnx: |
|
350 photo1 = cnx.entity_from_eid(photo1.eid) |
|
351 self.failUnless(photo1.may_be_read_by) |
|
352 # test security with permissions |
|
353 self.assertEquals(2, len(cnx.execute('File X'))) # now toto has access to photo2 |
|
354 self.assertEquals(1, len(cnx.execute('Folder X'))) # and to restricted folder |
|
355 |
|
356 if __name__ == '__main__': |
|
357 from logilab.common.testlib import unittest_main |
|
358 unittest_main() |
|
359 |
|
360 It's not complete, but shows most things you'll want to do in tests: adding some |
|
361 content, creating users and connecting as them in the test, etc... |
|
362 |
|
363 To run it type: |
|
364 |
|
365 .. sourcecode:: bash |
|
366 |
|
367 $ pytest unittest_sytweb.py |
|
368 ======================== unittest_sytweb.py ======================== |
|
369 -> creating tables [....................] |
|
370 -> inserting default user and default groups. |
|
371 -> storing the schema in the database [....................] |
|
372 -> database for instance data initialized. |
|
373 . |
|
374 ---------------------------------------------------------------------- |
|
375 Ran 1 test in 22.547s |
|
376 |
|
377 OK |
|
378 |
|
379 |
|
380 The first execution is taking time, since it creates a sqlite database for the |
|
381 test instance. The second one will be much quicker: |
|
382 |
|
383 .. sourcecode:: bash |
|
384 |
|
385 $ pytest unittest_sytweb.py |
|
386 ======================== unittest_sytweb.py ======================== |
|
387 . |
|
388 ---------------------------------------------------------------------- |
|
389 Ran 1 test in 2.662s |
|
390 |
|
391 OK |
|
392 |
|
393 If you do some changes in your schema, you'll have to force regeneration of that |
|
394 database. You do that by removing the tmpdb files before running the test: :: |
|
395 |
|
396 $ rm data/database/tmpdb* |
|
397 |
|
398 |
|
399 .. Note:: |
|
400 pytest is a very convenient utility used to control test execution. It is available from the `logilab-common`_ package. |
|
401 |
|
402 .. _`logilab-common`: http://www.logilab.org/project/logilab-common |
|
403 |
|
404 .. _adv_tuto_migration_script: |
|
405 |
|
406 Step 4: writing the migration script and migrating the instance |
|
407 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
408 |
|
409 Prior to those changes, I created an instance, fed it with some data, so I |
|
410 don't want to create a new one, but to migrate the existing one. Let's see how to |
|
411 do that. |
|
412 |
|
413 Migration commands should be put in the cube's *migration* directory, in a |
|
414 file named file:`<X.Y.Z>_Any.py` ('Any' being there mostly for historical reasons). |
|
415 |
|
416 Here I'll create a *migration/0.2.0_Any.py* file containing the following |
|
417 instructions: |
|
418 |
|
419 .. sourcecode:: python |
|
420 |
|
421 add_relation_type('may_be_read_by') |
|
422 add_relation_type('visibility') |
|
423 sync_schema_props_perms() |
|
424 |
|
425 Then I update the version number in the cube's *__pkginfo__.py* to 0.2.0. And |
|
426 that's it! Those instructions will: |
|
427 |
|
428 * update the instance's schema by adding our two new relations and update the |
|
429 underlying database tables accordingly (the first two instructions) |
|
430 |
|
431 * update schema's permissions definition (the last instruction) |
|
432 |
|
433 |
|
434 To migrate my instance I simply type:: |
|
435 |
|
436 cubicweb-ctl upgrade sytweb_instance |
|
437 |
|
438 You'll then be asked some questions to do the migration step by step. You should say |
|
439 YES when it asks if a backup of your database should be done, so you can get back |
|
440 to initial state if anything goes wrong... |
|