doc/book/en/annexes/rql/implementation.rst
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Wed, 03 Feb 2010 18:44:24 +0100
changeset 4437 21f2e01fdd6a
parent 2172 cf8f9180e63e
child 4446 a413fac5ff5e
permissions -rw-r--r--
update exemples using the 3.6 api and add/fix some sections (schema, vreg, talk about CW_MODE in concepts...). So much to do :'(
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1714
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     1
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     2
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     3
Implementation
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     4
--------------
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     5
BNF grammar
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     6
~~~~~~~~~~~
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     7
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     8
The terminal elements are in capital letters, non-terminal in lowercase.
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
     9
The value of the terminal elements (between quotes) is a Python regular
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    10
expression.
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    11
::
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    12
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    13
     statement:: = (select | delete | insert | update) ';'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    14
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    15
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    16
     # select specific rules
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    17
     select      ::= 'DISTINCT'? E_TYPE selected_terms restriction? group? sort?
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    18
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    19
     selected_terms ::= expression ( ',' expression)*
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    20
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    21
     group       ::= 'GROUPBY' VARIABLE ( ',' VARIABLE)*
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    22
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    23
     sort        ::= 'ORDERBY' sort_term ( ',' sort_term)*
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    24
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    25
     sort_term   ::=  VARIABLE sort_method =?
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    26
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    27
     sort_method ::= 'ASC' | 'DESC'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    28
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    29
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    30
     # delete specific rules
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    31
     delete ::= 'DELETE' (variables_declaration | relations_declaration) restriction?
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    32
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    33
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    34
     # insert specific rules
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    35
     insert ::= 'INSERT' variables_declaration ( ':' relations_declaration)? restriction?
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    36
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    37
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    38
     # update specific rules
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    39
     update ::= 'SET' relations_declaration restriction
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    40
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    41
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    42
     # common rules
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    43
     variables_declaration ::= E_TYPE VARIABLE (',' E_TYPE VARIABLE)*
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    44
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    45
     relations_declaration ::= simple_relation (',' simple_relation)*
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    46
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    47
     simple_relation ::= VARIABLE R_TYPE expression
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    48
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    49
     restriction ::= 'WHERE' relations
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    50
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    51
     relations   ::= relation (LOGIC_OP relation)*
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    52
                   | '(' relations')'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    53
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    54
     relation    ::= 'NOT'? VARIABLE R_TYPE COMP_OP? expression
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    55
                   | 'NOT'? R_TYPE VARIABLE 'IN' '(' expression (',' expression)* ')'
2172
cf8f9180e63e delete-trailing-whitespace
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 1714
diff changeset
    56
1714
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    57
     expression  ::= var_or_func_or_const (MATH_OP var_or_func_or_const) *
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    58
                   | '(' expression ')'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    59
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    60
     var_or_func_or_const ::= VARIABLE | function | constant
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    61
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    62
     function    ::= FUNCTION '(' expression ( ',' expression) * ')'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    63
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    64
     constant    ::= KEYWORD | STRING | FLOAT | INT
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    65
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    66
     # tokens
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    67
     LOGIC_OP ::= ',' | 'OR' | 'AND'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    68
     MATH_OP  ::= '+' | '-' | '/' | '*'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    69
     COMP_OP  ::= '>' | '>=' | '=' | '<=' | '<' | '~=' | 'LIKE'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    70
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    71
     FUNCTION ::= 'MIN' | 'MAX' | 'SUM' | 'AVG' | 'COUNT' | 'UPPER' | 'LOWER'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    72
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    73
     VARIABLE ::= '[A-Z][A-Z0-9]*'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    74
     E_TYPE   ::= '[A-Z]\w*'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    75
     R_TYPE   ::= '[a-z_]+'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    76
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    77
     KEYWORD  ::= 'TRUE' | 'FALSE' | 'NULL' | 'TODAY' | 'NOW'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    78
     STRING   ::= "'([^'\]|\\.)*'" |'"([^\"]|\\.)*\"'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    79
     FLOAT    ::= '\d+\.\d*'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    80
     INT      ::= '\d+'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    81
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    82
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    83
Internal representation (syntactic tree)
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    84
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    85
2172
cf8f9180e63e delete-trailing-whitespace
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 1714
diff changeset
    86
The tree research does not contain the selected variables
1714
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    87
(e.g. there is only what follows "WHERE").
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    88
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    89
The insertion tree does not contain the variables inserted or relations
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    90
defined on these variables (e.g. there is only what follows "WHERE").
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    91
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    92
The removal tree does not contain the deleted variables and relations
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    93
(e.g. there is only what follows the "WHERE").
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    94
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    95
The update tree does not contain the variables and relations updated
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    96
(e.g. there is only what follows the "WHERE").
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    97
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    98
::
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
    99
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   100
     Select         ((Relationship | And | Or)?, Group?, Sort?)
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   101
     Insert         (Relations | And | Or)?
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   102
     Delete         (Relationship | And | Or)?
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   103
     Update         (Relations | And | Or)?
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   104
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   105
     And            ((Relationship | And | Or), (Relationship | And | Or))
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   106
     Or             ((Relationship | And | Or), (Relationship | And | Or))
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   107
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   108
     Relationship   ((VariableRef, Comparison))
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   109
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   110
     Comparison     ((Function | MathExpression | Keyword | Constant | VariableRef) +)
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   111
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   112
     Function       (())
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   113
     MathExpression ((MathExpression | Keyword | Constant | VariableRef), (MathExpression | Keyword | Constant | VariableRef))
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   114
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   115
     Group          (VariableRef +)
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   116
     Sort           (SortTerm +)
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   117
     SortTerm       (VariableRef +)
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   118
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   119
     VariableRef    ()
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   120
     Variable       ()
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   121
     Keyword        ()
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   122
     Constant       ()
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   123
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   124
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   125
Known limitations
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   126
~~~~~~~~~~~~~~~~~
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   127
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   128
- The current implementation does not support linking two relations of type 'is'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   129
  with a OR. I do not think that the negation is supported on this type of
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   130
  relation (XXX FIXME to be confirmed).
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   131
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   132
- Relations defining the variables must be left to those using them.  For
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   133
  example::
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   134
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   135
     Point P where P abs X, P ord Y, P value X+Y
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   136
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   137
  is valid, but::
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   138
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   139
     Point P where P abs X, P value X+Y, P ord Y
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   140
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   141
  is not.
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   142
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   143
- missing proper explicit type conversion,  COALESCE and certainly other things...
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   144
4437
21f2e01fdd6a update exemples using the 3.6 api and add/fix some sections (schema, vreg, talk about CW_MODE in concepts...). So much to do :'(
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2172
diff changeset
   145
- writing a rql query._cwuire knowledge of the schema used (with real relation
1714
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   146
  names and entities, not those viewing in the user interface). On the other
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   147
  hand, we can not really bypass that, and it is the job of a user interface to
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   148
  hide the RQL.
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   149
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   150
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   151
Topics
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   152
~~~~~~
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   153
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   154
It would be convenient to express the schema matching
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   155
relations (non-recursive rules)::
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   156
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   157
     Document class Type <-> Document occurence_of Fiche class Type
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   158
     Sheet class Type    <-> Form collection Collection class Type
2172
cf8f9180e63e delete-trailing-whitespace
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 1714
diff changeset
   159
1714
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   160
Therefore 1. becomes::
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   161
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   162
     Document X where
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   163
     X class C, C name 'Cartoon'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   164
     X owned_by U, U login 'syt'
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   165
     X available true
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   166
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   167
I'm not sure that we should handle this at RQL level ...
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   168
a721966779be new book layout, do not compile yet
sylvain.thenault@logilab.fr
parents:
diff changeset
   169
There should also be a special relation 'anonymous'.