server/sources/storages.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 19 May 2011 10:53:11 +0200
changeset 7398 26695dd703d8
parent 7118 e094b3d4eb95
child 7454 1090724f28ed
permissions -rw-r--r--
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session Also document session's data storage and some other internals. Hopefuly things will get clearer. Closes #1684860: vocabulary confusion in repository code: notion of 'pool'
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7118
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
"""custom storages for the system source"""
5693
8af6623f3d4e [pylint] fix detected name errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5625
diff changeset
    19
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
from os import unlink, path as osp
6383
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
    21
from contextlib import contextmanager
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    23
from yams.schema import role_name
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    24
5693
8af6623f3d4e [pylint] fix detected name errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5625
diff changeset
    25
from cubicweb import Binary, ValidationError
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
    26
from cubicweb.server import hook
7118
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
    27
from cubicweb.server.edition import EditedEntity
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
    28
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    29
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
def set_attribute_storage(repo, etype, attr, storage):
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4831
diff changeset
    31
    repo.system_source.set_storage(etype, attr, storage)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    32
4512
e7ac20bf3629 unset_attribute_storage, for testing purpose at least
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4349
diff changeset
    33
def unset_attribute_storage(repo, etype, attr):
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4831
diff changeset
    34
    repo.system_source.unset_storage(etype, attr)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    35
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
    36
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    37
class Storage(object):
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    38
    """abstract storage
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    39
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    40
    * If `source_callback` is true (by default), the callback will be run during
5625
6ee2a7b6f194 [external storage] refactor to give session to storage's callback (needed by vcsfile storage)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5599
diff changeset
    41
      query result process of fetched attribute's value and should have the
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    42
      following prototype::
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    43
5625
6ee2a7b6f194 [external storage] refactor to give session to storage's callback (needed by vcsfile storage)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5599
diff changeset
    44
        callback(self, source, session, value)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    45
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    46
      where `value` is the value actually stored in the backend. None values
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    47
      will be skipped (eg callback won't be called).
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    48
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    49
    * if `source_callback` is false, the callback will be run during sql
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    50
      generation when some attribute with a custom storage is accessed and
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    51
      should have the following prototype::
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    52
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    53
        callback(self, generator, relation, linkedvar)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    54
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    55
      where `generator` is the sql generator, `relation` the current rql syntax
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    56
      tree relation and linkedvar the principal syntax tree variable holding the
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    57
      attribute.
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    58
    """
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    59
    is_source_callback = True
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    60
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    61
    def callback(self, *args):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    62
        """see docstring for prototype, which vary according to is_source_callback
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    63
        """
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    64
        raise NotImplementedError()
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    65
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    66
    def entity_added(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    67
        """an entity using this storage for attr has been added"""
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    68
        raise NotImplementedError()
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    69
    def entity_updated(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    70
        """an entity using this storage for attr has been updatded"""
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    71
        raise NotImplementedError()
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    72
    def entity_deleted(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    73
        """an entity using this storage for attr has been deleted"""
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    74
        raise NotImplementedError()
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
    75
    def migrate_entity(self, entity, attribute):
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
    76
        """migrate an entity attribute to the storage"""
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
    77
        raise NotImplementedError()
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    78
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    79
# TODO
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    80
# * make it configurable without code
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    81
# * better file path attribution
4329
815e08c53548 add a reminder
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4322
diff changeset
    82
# * handle backup/restore
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    83
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    84
def uniquify_path(dirpath, basename):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    85
    """return a unique file name for `basename` in `dirpath`, or None
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    86
    if all attemps failed.
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    87
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    88
    XXX subject to race condition.
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    89
    """
5549
ab3a69a34626 [bfss] ensure base name doesn't contain path separator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    90
    path = osp.join(dirpath, basename.replace(osp.sep, '-'))
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    91
    if not osp.isfile(path):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    92
        return path
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    93
    base, ext = osp.splitext(path)
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    94
    for i in xrange(1, 256):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    95
        path = '%s%s%s' % (base, i, ext)
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    96
        if not osp.isfile(path):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    97
            return path
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    98
    return None
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    99
6383
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   100
@contextmanager
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   101
def fsimport(session):
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   102
    present = 'fs_importing' in session.transaction_data
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   103
    old_value = session.transaction_data.get('fs_importing')
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   104
    session.transaction_data['fs_importing'] = True
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   105
    yield
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   106
    if present:
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   107
        session.transaction_data['fs_importing'] = old_value
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   108
    else:
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   109
        del session.transaction_data['fs_importing']
19ebe0b994d6 Add a fsimport context manage to localy enable fsimporting.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5863
diff changeset
   110
5398
b9e1abe1bdfe [repo] cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5397
diff changeset
   111
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   112
class BytesFileSystemStorage(Storage):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   113
    """store Bytes attribute value on the file system"""
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   114
    def __init__(self, defaultdir, fsencoding='utf-8'):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   115
        self.default_directory = defaultdir
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   116
        self.fsencoding = fsencoding
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   117
5625
6ee2a7b6f194 [external storage] refactor to give session to storage's callback (needed by vcsfile storage)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5599
diff changeset
   118
    def callback(self, source, session, value):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   119
        """sql generator callback when some attribute with a custom storage is
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   120
        accessed
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   121
        """
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
   122
        fpath = source.binary_to_str(value)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
   123
        try:
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   124
            return Binary(file(fpath, 'rb').read())
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
   125
        except OSError, ex:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
   126
            source.critical("can't open %s: %s", value, ex)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
   127
            return None
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   128
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   129
    def entity_added(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   130
        """an entity using this storage for attr has been added"""
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   131
        if entity._cw.transaction_data.get('fs_importing'):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   132
            binary = Binary(file(entity.cw_edited[attr].getvalue(), 'rb').read())
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   133
        else:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   134
            binary = entity.cw_edited.pop(attr)
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   135
            fpath = self.new_fs_path(entity, attr)
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   136
            # bytes storage used to store file's path
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   137
            entity.cw_edited.edited_attribute(attr, Binary(fpath))
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   138
            file(fpath, 'wb').write(binary.getvalue())
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   139
            AddFileOp.get_instance(entity._cw).add_data(fpath)
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   140
        return binary
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   141
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   142
    def entity_updated(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   143
        """an entity using this storage for attr has been updatded"""
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   144
        # get the name of the previous file containing the value
5599
be94157bd754 [bfss] Rename filenames according to their metadata on entity update.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5549
diff changeset
   145
        oldpath = self.current_fs_path(entity, attr)
5183
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   146
        if entity._cw.transaction_data.get('fs_importing'):
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   147
            # If we are importing from the filesystem, the file already exists.
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   148
            # We do not need to create it but we need to fetch the content of
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   149
            # the file as the actual content of the attribute
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   150
            fpath = entity.cw_edited[attr].getvalue()
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   151
            binary = Binary(file(fpath, 'rb').read())
5183
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   152
        else:
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   153
            # We must store the content of the attributes
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   154
            # into a file to stay consistent with the behaviour of entity_add.
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   155
            # Moreover, the BytesFileSystemStorage expects to be able to
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   156
            # retrieve the current value of the attribute at anytime by reading
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   157
            # the file on disk. To be able to rollback things, use a new file
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   158
            # and keep the old one that will be removed on commit if everything
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   159
            # went ok.
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   160
            #
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   161
            # fetch the current attribute value in memory
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   162
            binary = entity.cw_edited.pop(attr)
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   163
            # Get filename for it
5599
be94157bd754 [bfss] Rename filenames according to their metadata on entity update.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5549
diff changeset
   164
            fpath = self.new_fs_path(entity, attr)
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   165
            assert not osp.exists(fpath)
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   166
            # write attribute value on disk
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   167
            file(fpath, 'wb').write(binary.getvalue())
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   168
            # Mark the new file as added during the transaction.
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   169
            # The file will be removed on rollback
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   170
            AddFileOp.get_instance(entity._cw).add_data(fpath)
5599
be94157bd754 [bfss] Rename filenames according to their metadata on entity update.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5549
diff changeset
   171
        if oldpath != fpath:
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   172
            # register the new location for the file.
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   173
            entity.cw_edited.edited_attribute(attr, Binary(fpath))
5857
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   174
            # Mark the old file as useless so the file will be removed at
1a24c62aefc5 [bfss] fix file update to ensure file's content is available on the fs asap...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5693
diff changeset
   175
            # commit.
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   176
            DeleteFileOp.get_instance(entity._cw).add_data(oldpath)
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   177
        return binary
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   178
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   179
    def entity_deleted(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   180
        """an entity using this storage for attr has been deleted"""
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   181
        fpath = self.current_fs_path(entity, attr)
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   182
        DeleteFileOp.get_instance(entity._cw).add_data(fpath)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   183
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   184
    def new_fs_path(self, entity, attr):
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   185
        # We try to get some hint about how to name the file using attribute's
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   186
        # name metadata, so we use the real file name and extension when
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   187
        # available. Keeping the extension is useful for example in the case of
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   188
        # PIL processing that use filename extension to detect content-type, as
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   189
        # well as providing more understandable file names on the fs.
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   190
        basename = [str(entity.eid), attr]
5576
08c6d4d6c50c [deprecation] fix 3.9 deprecation warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5549
diff changeset
   191
        name = entity.cw_attr_metadata(attr, 'name')
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   192
        if name is not None:
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   193
            basename.append(name.encode(self.fsencoding))
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   194
        fspath = uniquify_path(self.default_directory, '_'.join(basename))
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   195
        if fspath is None:
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   196
            msg = entity._cw._('failed to uniquify path (%s, %s)') % (
5693
8af6623f3d4e [pylint] fix detected name errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5625
diff changeset
   197
                self.default_directory, '_'.join(basename))
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   198
            raise ValidationError(entity.eid, {role_name(attr, 'subject'): msg})
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   199
        return fspath
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   200
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   201
    def current_fs_path(self, entity, attr):
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7118
diff changeset
   202
        sysource = entity._cw.cnxset.source('system')
4349
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
   203
        cu = sysource.doexec(entity._cw,
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
   204
                             'SELECT cw_%s FROM cw_%s WHERE cw_eid=%s' % (
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   205
                             attr, entity.__regid__, entity.eid))
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   206
        rawvalue = cu.fetchone()[0]
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   207
        if rawvalue is None: # no previous value
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   208
            return self.new_fs_path(entity, attr)
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   209
        return sysource._process_value(rawvalue, cu.description[0],
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   210
                                       binarywrap=str)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   211
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   212
    def migrate_entity(self, entity, attribute):
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   213
        """migrate an entity attribute to the storage"""
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   214
        entity.cw_edited = EditedEntity(entity, **entity.cw_attr_cache)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   215
        self.entity_added(entity, attribute)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   216
        session = entity._cw
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   217
        source = session.repo.system_source
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   218
        attrs = source.preprocess_entity(entity)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   219
        sql = source.sqlgen.update('cw_' + entity.__regid__, attrs,
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   220
                                   ['cw_eid'])
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5396
diff changeset
   221
        source.doexec(session, sql, attrs)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   222
        entity.cw_edited = None
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   223
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   224
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   225
class AddFileOp(hook.DataOperationMixIn, hook.Operation):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   226
    def rollback_event(self):
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   227
        for filepath in self.get_data():
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   228
            try:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   229
                unlink(filepath)
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   230
            except Exception, ex:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   231
                self.error('cant remove %s: %s' % (filepath, ex))
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   232
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   233
class DeleteFileOp(hook.DataOperationMixIn, hook.Operation):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   234
    def postcommit_event(self):
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   235
        for filepath in self.get_data():
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   236
            try:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   237
                unlink(filepath)
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   238
            except Exception, ex:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   239
                self.error('cant remove %s: %s' % (filepath, ex))