server/sources/storages.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Fri, 23 Apr 2010 17:54:34 +0200
changeset 5396 78d92a47a4e5
parent 5219 35d44017c72b
child 5397 cdbf823450aa
permissions -rw-r--r--
[bfss] use set_operation for AddFileOp/DeleteFileOp
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     1
"""custom storages for the system source"""
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
from os import unlink, path as osp
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
     4
from yams.schema import role_name
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
     5
4349
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
     6
from cubicweb import Binary
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
     7
from cubicweb.server import hook
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
def set_attribute_storage(repo, etype, attr, storage):
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4831
diff changeset
    10
    repo.system_source.set_storage(etype, attr, storage)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
4512
e7ac20bf3629 unset_attribute_storage, for testing purpose at least
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4349
diff changeset
    12
def unset_attribute_storage(repo, etype, attr):
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4831
diff changeset
    13
    repo.system_source.unset_storage(etype, attr)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    14
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
class Storage(object):
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    16
    """abstract storage
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    17
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    18
    * If `source_callback` is true (by default), the callback will be run during
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    19
      query result process of fetched attribute's valu and should have the
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    20
      following prototype::
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    21
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    22
        callback(self, source, value)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    23
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    24
      where `value` is the value actually stored in the backend. None values
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    25
      will be skipped (eg callback won't be called).
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    26
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    27
    * if `source_callback` is false, the callback will be run during sql
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    28
      generation when some attribute with a custom storage is accessed and
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    29
      should have the following prototype::
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    30
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    31
        callback(self, generator, relation, linkedvar)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    32
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    33
      where `generator` is the sql generator, `relation` the current rql syntax
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    34
      tree relation and linkedvar the principal syntax tree variable holding the
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    35
      attribute.
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    36
    """
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    37
    is_source_callback = True
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    38
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    39
    def callback(self, *args):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    40
        """see docstring for prototype, which vary according to is_source_callback
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    41
        """
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    42
        raise NotImplementedError()
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    43
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    44
    def entity_added(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    45
        """an entity using this storage for attr has been added"""
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    46
        raise NotImplementedError()
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    47
    def entity_updated(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    48
        """an entity using this storage for attr has been updatded"""
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    49
        raise NotImplementedError()
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    50
    def entity_deleted(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    51
        """an entity using this storage for attr has been deleted"""
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    52
        raise NotImplementedError()
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    53
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    54
# TODO
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    55
# * make it configurable without code
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    56
# * better file path attribution
4329
815e08c53548 add a reminder
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4322
diff changeset
    57
# * handle backup/restore
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    58
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    59
def uniquify_path(dirpath, basename):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    60
    """return a unique file name for `basename` in `dirpath`, or None
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    61
    if all attemps failed.
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    62
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    63
    XXX subject to race condition.
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    64
    """
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    65
    path = osp.join(dirpath, basename)
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    66
    if not osp.isfile(path):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    67
        return path
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    68
    base, ext = osp.splitext(path)
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    69
    for i in xrange(1, 256):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    70
        path = '%s%s%s' % (base, i, ext)
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    71
        if not osp.isfile(path):
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    72
            return path
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    73
    return None
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
    74
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    75
class BytesFileSystemStorage(Storage):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    76
    """store Bytes attribute value on the file system"""
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
    77
    def __init__(self, defaultdir, fsencoding='utf-8'):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    78
        self.default_directory = defaultdir
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
    79
        self.fsencoding = fsencoding
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    80
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    81
    def callback(self, source, value):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    82
        """sql generator callback when some attribute with a custom storage is
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    83
        accessed
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    84
        """
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    85
        fpath = source.binary_to_str(value)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    86
        try:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    87
            return Binary(file(fpath).read())
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    88
        except OSError, ex:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    89
            source.critical("can't open %s: %s", value, ex)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5012
diff changeset
    90
            return None
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    91
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    92
    def entity_added(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    93
        """an entity using this storage for attr has been added"""
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
    94
        if entity._cw.transaction_data.get('fs_importing'):
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
    95
            binary = Binary(file(entity[attr].getvalue()).read())
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
    96
        else:
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
    97
            binary = entity.pop(attr)
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
    98
            fpath = self.new_fs_path(entity, attr)
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
    99
            # bytes storage used to store file's path
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   100
            entity[attr] = Binary(fpath)
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   101
            file(fpath, 'w').write(binary.getvalue())
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   102
            hook.set_operation(entity._cw, 'bfss_added', fpath, AddFileOp)
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   103
        return binary
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   104
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   105
    def entity_updated(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   106
        """an entity using this storage for attr has been updatded"""
5183
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   107
        if entity._cw.transaction_data.get('fs_importing'):
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   108
            oldpath = self.current_fs_path(entity, attr)
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   109
            fpath = entity[attr].getvalue()
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   110
            if oldpath != fpath:
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   111
                hook.set_operation(entity._cw, 'bfss_deleted', oldpath,
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   112
                                   DeleteFileOp)
5183
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   113
            binary = Binary(file(fpath).read())
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   114
        else:
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   115
            binary = entity.pop(attr)
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   116
            fpath = self.current_fs_path(entity, attr)
8d66003351f8 [storage] consider fs_importing on update operations too
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5131
diff changeset
   117
            UpdateFileOp(entity._cw, filepath=fpath, filedata=binary.getvalue())
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   118
        return binary
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   119
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   120
    def entity_deleted(self, entity, attr):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   121
        """an entity using this storage for attr has been deleted"""
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   122
        fpath = self.current_fs_path(entity, attr)
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   123
        hook.set_operation(entity._cw, 'bfss_deleted', fpath, DeleteFileOp)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   124
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   125
    def new_fs_path(self, entity, attr):
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   126
        # We try to get some hint about how to name the file using attribute's
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   127
        # name metadata, so we use the real file name and extension when
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   128
        # available. Keeping the extension is useful for example in the case of
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   129
        # PIL processing that use filename extension to detect content-type, as
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   130
        # well as providing more understandable file names on the fs.
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   131
        basename = [str(entity.eid), attr]
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   132
        name = entity.attr_metadata(attr, 'name')
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   133
        if name is not None:
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   134
            basename.append(name.encode(self.fsencoding))
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   135
        fspath = uniquify_path(self.default_directory, '_'.join(basename))
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   136
        if fspath is None:
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   137
            msg = entity._cw._('failed to uniquify path (%s, %s)') % (
5219
35d44017c72b [storage] missing qrefresh in previous patch applied: fix comment, error message, and use a storage specified encoding, not cubicweb's encoding
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5218
diff changeset
   138
                dirpath, '_'.join(basename))
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   139
            raise ValidationError(entity.eid, {role_name(attr, 'subject'): msg})
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   140
        return fspath
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   141
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   142
    def current_fs_path(self, entity, attr):
4349
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
   143
        sysource = entity._cw.pool.source('system')
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
   144
        cu = sysource.doexec(entity._cw,
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
   145
                             'SELECT cw_%s FROM cw_%s WHERE cw_eid=%s' % (
5218
aebd00a2d316 [fix] fix path unicity process in BytesFileSystemStorage.new_fs_path
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5183
diff changeset
   146
                             attr, entity.__regid__, entity.eid))
5131
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   147
        rawvalue = cu.fetchone()[0]
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   148
        if rawvalue is None: # no previous value
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   149
            return self.new_fs_path(entity, attr)
88b5ca8da928 [storages] fix fs_importing side-effect on entity.data
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5013
diff changeset
   150
        return sysource._process_value(rawvalue, cu.description[0],
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   151
                                       binarywrap=str)
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   152
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   153
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   154
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   155
class AddFileOp(hook.Operation):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   156
    def rollback_event(self):
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   157
        for filepath in self.session.transaction_data.pop('bfss_added'):
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   158
            try:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   159
                unlink(filepath)
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   160
            except Exception, ex:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   161
                self.error('cant remove %s: %s' % (filepath, ex))
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   162
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   163
class DeleteFileOp(hook.Operation):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   164
    def commit_event(self):
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   165
        for filepath in self.session.transaction_data.pop('bfss_deleted'):
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   166
            try:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   167
                unlink(filepath)
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   168
            except Exception, ex:
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   169
                self.error('cant remove %s: %s' % (filepath, ex))
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   170
5396
78d92a47a4e5 [bfss] use set_operation for AddFileOp/DeleteFileOp
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5219
diff changeset
   171
class UpdateFileOp(hook.Operation):
4322
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   172
    def precommit_event(self):
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   173
        try:
f65743cc53e4 first draft for a simple hooks based custom attribute storage,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   174
            file(self.filepath, 'w').write(self.filedata)
4349
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
   175
        except Exception, ex:
48dadeeacfa5 [bfss] make it works when adding/updating entities with an attribute using bfss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4329
diff changeset
   176
            self.exception(str(ex))