[CalendarServer-changes] [14521] twext/trunk
source_changes at macosforge.org
source_changes at macosforge.org
Fri Mar 6 13:39:59 PST 2015
Revision: 14521
http://trac.calendarserver.org//changeset/14521
Author: cdaboo at apple.com
Date: 2015-03-06 13:39:59 -0800 (Fri, 06 Mar 2015)
Log Message:
-----------
Merge pod-2-pod branch to trunk.
Modified Paths:
--------------
twext/trunk/twext/enterprise/dal/record.py
twext/trunk/twext/enterprise/dal/syntax.py
twext/trunk/twext/enterprise/dal/test/test_record.py
twext/trunk/twext/enterprise/dal/test/test_sqlsyntax.py
Property Changed:
----------------
twext/trunk/
twext/trunk/twext/
Property changes on: twext/trunk
___________________________________________________________________
Modified: svn:mergeinfo
- /twext/branches/users/cdaboo/jobqueue-3:13444-13471
/twext/branches/users/cdaboo/jobs:12742-12780
/twext/branches/users/sagen/recordtypes:13647-13658
/twext/branches/users/sagen/recordtypes-2:13659
+ /twext/branches/users/cdaboo/jobqueue-3:13444-13471
/twext/branches/users/cdaboo/jobs:12742-12780
/twext/branches/users/cdaboo/pod2pod-migration:14416-14520
/twext/branches/users/sagen/recordtypes:13647-13658
/twext/branches/users/sagen/recordtypes-2:13659
Property changes on: twext/trunk/twext
___________________________________________________________________
Modified: svn:mergeinfo
- /CalDAVTester/trunk/twext:11193-11198
/CalendarServer/branches/config-separation/twext:4379-4443
/CalendarServer/branches/egg-info-351/twext:4589-4625
/CalendarServer/branches/generic-sqlstore/twext:6167-6191
/CalendarServer/branches/new-store/twext:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/twext:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/twext:5936-5981
/CalendarServer/branches/release/CalendarServer-4.3-dev/twext:10180-10190,10192
/CalendarServer/branches/release/CalendarServer-5.1-dev/twext:11846
/CalendarServer/branches/users/cdaboo/batchupload-6699/twext:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/twext:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes/twext:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/twext:3628-3644
/CalendarServer/branches/users/cdaboo/fix-no-ischedule/twext:11607-11871
/CalendarServer/branches/users/cdaboo/implicituidrace/twext:8137-8141
/CalendarServer/branches/users/cdaboo/ischedule-dkim/twext:9747-9979
/CalendarServer/branches/users/cdaboo/json/twext:11622-11912
/CalendarServer/branches/users/cdaboo/managed-attachments/twext:9985-10145
/CalendarServer/branches/users/cdaboo/more-sharing-5591/twext:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/twext:4465-4957
/CalendarServer/branches/users/cdaboo/performance-tweaks/twext:11824-11836
/CalendarServer/branches/users/cdaboo/pods/twext:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar/twext:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/twext:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes/twext:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/twext:5071-5105
/CalendarServer/branches/users/cdaboo/reverse-proxy-pods/twext:11875-11900
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext:5188-5440
/CalendarServer/branches/users/cdaboo/sharing-in-the-store/twext:11935-12016
/CalendarServer/branches/users/cdaboo/store-scheduling/twext:10876-11129
/CalendarServer/branches/users/cdaboo/timezones/twext:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging/twext:8730-8743
/CalendarServer/branches/users/gaya/sharedgroups-3/twext:11088-11204
/CalendarServer/branches/users/glyph/always-abort-txn-on-error/twext:9958-9969
/CalendarServer/branches/users/glyph/case-insensitive-uid/twext:8772-8805
/CalendarServer/branches/users/glyph/conn-limit/twext:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/twext:4971-5080
/CalendarServer/branches/users/glyph/dalify/twext:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect/twext:6824-6876
/CalendarServer/branches/users/glyph/deploybuild/twext:7563-7572
/CalendarServer/branches/users/glyph/digest-auth-redux/twext:10624-10635
/CalendarServer/branches/users/glyph/disable-quota/twext:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres/twext:6592-6614
/CalendarServer/branches/users/glyph/enforce-max-requests/twext:11640-11643
/CalendarServer/branches/users/glyph/hang-fix/twext:11465-11491
/CalendarServer/branches/users/glyph/imip-and-admin-html/twext:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client/twext:9054-9105
/CalendarServer/branches/users/glyph/launchd-wrapper-bis/twext:11413-11436
/CalendarServer/branches/users/glyph/linux-tests/twext:6893-6900
/CalendarServer/branches/users/glyph/log-cleanups/twext:11691-11731
/CalendarServer/branches/users/glyph/migrate-merge/twext:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes/twext:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6/twext:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7/twext:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete/twext:8321-8330
/CalendarServer/branches/users/glyph/new-export/twext:7444-7485
/CalendarServer/branches/users/glyph/one-home-list-api/twext:10048-10073
/CalendarServer/branches/users/glyph/oracle/twext:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls/twext:7340-7351
/CalendarServer/branches/users/glyph/other-html/twext:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim/twext:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade/twext:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1/twext:8571-8583
/CalendarServer/branches/users/glyph/q/twext:9560-9688
/CalendarServer/branches/users/glyph/queue-locking-and-timing/twext:10204-10289
/CalendarServer/branches/users/glyph/quota/twext:7604-7637
/CalendarServer/branches/users/glyph/sendfdport/twext:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes/twext:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2/twext:8155-8174
/CalendarServer/branches/users/glyph/sharedpool/twext:6490-6550
/CalendarServer/branches/users/glyph/sharing-api/twext:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones/twext:8524-8535
/CalendarServer/branches/users/glyph/sql-store/twext:5929-6073
/CalendarServer/branches/users/glyph/start-service-start-loop/twext:11060-11065
/CalendarServer/branches/users/glyph/subtransactions/twext:7248-7258
/CalendarServer/branches/users/glyph/table-alias/twext:8651-8664
/CalendarServer/branches/users/glyph/uidexport/twext:7673-7676
/CalendarServer/branches/users/glyph/unshare-when-access-revoked/twext:10562-10595
/CalendarServer/branches/users/glyph/use-system-twisted/twext:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize/twext:9268-9296
/CalendarServer/branches/users/glyph/warning-cleanups/twext:11347-11357
/CalendarServer/branches/users/glyph/whenNotProposed/twext:11881-11897
/CalendarServer/branches/users/glyph/xattrs-from-files/twext:7757-7769
/CalendarServer/branches/users/sagen/applepush/twext:8126-8184
/CalendarServer/branches/users/sagen/inboxitems/twext:7380-7381
/CalendarServer/branches/users/sagen/locations-resources/twext:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/twext:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events/twext:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/twext:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/twext:4068-4075
/CalendarServer/branches/users/sagen/resources-2/twext:5084-5093
/CalendarServer/branches/users/sagen/testing/twext:10827-10851,10853-10855
/CalendarServer/branches/users/wsanchez/transations/twext:5515-5593
/twext/branches/users/cdaboo/jobqueue-3/twext:13444-13471
/twext/branches/users/cdaboo/jobs/twext:12742-12780
/twext/branches/users/sagen/recordtypes/twext:13647-13658
/twext/branches/users/sagen/recordtypes-2/twext:13659
+ /CalDAVTester/trunk/twext:11193-11198
/CalendarServer/branches/config-separation/twext:4379-4443
/CalendarServer/branches/egg-info-351/twext:4589-4625
/CalendarServer/branches/generic-sqlstore/twext:6167-6191
/CalendarServer/branches/new-store-no-caldavfile-2/twext:5936-5981
/CalendarServer/branches/new-store-no-caldavfile/twext:5911-5935
/CalendarServer/branches/new-store/twext:5594-5934
/CalendarServer/branches/release/CalendarServer-4.3-dev/twext:10180-10190,10192
/CalendarServer/branches/release/CalendarServer-5.1-dev/twext:11846
/CalendarServer/branches/users/cdaboo/batchupload-6699/twext:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/twext:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes/twext:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/twext:3628-3644
/CalendarServer/branches/users/cdaboo/fix-no-ischedule/twext:11607-11871
/CalendarServer/branches/users/cdaboo/implicituidrace/twext:8137-8141
/CalendarServer/branches/users/cdaboo/ischedule-dkim/twext:9747-9979
/CalendarServer/branches/users/cdaboo/json/twext:11622-11912
/CalendarServer/branches/users/cdaboo/managed-attachments/twext:9985-10145
/CalendarServer/branches/users/cdaboo/more-sharing-5591/twext:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/twext:4465-4957
/CalendarServer/branches/users/cdaboo/performance-tweaks/twext:11824-11836
/CalendarServer/branches/users/cdaboo/pods/twext:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar/twext:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/twext:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes/twext:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/twext:5071-5105
/CalendarServer/branches/users/cdaboo/reverse-proxy-pods/twext:11875-11900
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext:5188-5440
/CalendarServer/branches/users/cdaboo/sharing-in-the-store/twext:11935-12016
/CalendarServer/branches/users/cdaboo/store-scheduling/twext:10876-11129
/CalendarServer/branches/users/cdaboo/timezones/twext:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging/twext:8730-8743
/CalendarServer/branches/users/gaya/sharedgroups-3/twext:11088-11204
/CalendarServer/branches/users/glyph/always-abort-txn-on-error/twext:9958-9969
/CalendarServer/branches/users/glyph/case-insensitive-uid/twext:8772-8805
/CalendarServer/branches/users/glyph/conn-limit/twext:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/twext:4971-5080
/CalendarServer/branches/users/glyph/dalify/twext:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect/twext:6824-6876
/CalendarServer/branches/users/glyph/deploybuild/twext:7563-7572
/CalendarServer/branches/users/glyph/digest-auth-redux/twext:10624-10635
/CalendarServer/branches/users/glyph/disable-quota/twext:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres/twext:6592-6614
/CalendarServer/branches/users/glyph/enforce-max-requests/twext:11640-11643
/CalendarServer/branches/users/glyph/hang-fix/twext:11465-11491
/CalendarServer/branches/users/glyph/imip-and-admin-html/twext:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client/twext:9054-9105
/CalendarServer/branches/users/glyph/launchd-wrapper-bis/twext:11413-11436
/CalendarServer/branches/users/glyph/linux-tests/twext:6893-6900
/CalendarServer/branches/users/glyph/log-cleanups/twext:11691-11731
/CalendarServer/branches/users/glyph/migrate-merge/twext:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes/twext:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6/twext:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7/twext:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete/twext:8321-8330
/CalendarServer/branches/users/glyph/new-export/twext:7444-7485
/CalendarServer/branches/users/glyph/one-home-list-api/twext:10048-10073
/CalendarServer/branches/users/glyph/oracle-nulls/twext:7340-7351
/CalendarServer/branches/users/glyph/oracle/twext:7106-7155
/CalendarServer/branches/users/glyph/other-html/twext:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim/twext:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade/twext:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1/twext:8571-8583
/CalendarServer/branches/users/glyph/q/twext:9560-9688
/CalendarServer/branches/users/glyph/queue-locking-and-timing/twext:10204-10289
/CalendarServer/branches/users/glyph/quota/twext:7604-7637
/CalendarServer/branches/users/glyph/sendfdport/twext:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes/twext:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2/twext:8155-8174
/CalendarServer/branches/users/glyph/sharedpool/twext:6490-6550
/CalendarServer/branches/users/glyph/sharing-api/twext:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones/twext:8524-8535
/CalendarServer/branches/users/glyph/sql-store/twext:5929-6073
/CalendarServer/branches/users/glyph/start-service-start-loop/twext:11060-11065
/CalendarServer/branches/users/glyph/subtransactions/twext:7248-7258
/CalendarServer/branches/users/glyph/table-alias/twext:8651-8664
/CalendarServer/branches/users/glyph/uidexport/twext:7673-7676
/CalendarServer/branches/users/glyph/unshare-when-access-revoked/twext:10562-10595
/CalendarServer/branches/users/glyph/use-system-twisted/twext:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize/twext:9268-9296
/CalendarServer/branches/users/glyph/warning-cleanups/twext:11347-11357
/CalendarServer/branches/users/glyph/whenNotProposed/twext:11881-11897
/CalendarServer/branches/users/glyph/xattrs-from-files/twext:7757-7769
/CalendarServer/branches/users/sagen/applepush/twext:8126-8184
/CalendarServer/branches/users/sagen/inboxitems/twext:7380-7381
/CalendarServer/branches/users/sagen/locations-resources-2/twext:5052-5061
/CalendarServer/branches/users/sagen/locations-resources/twext:5032-5051
/CalendarServer/branches/users/sagen/purge_old_events/twext:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/twext:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/twext:4068-4075
/CalendarServer/branches/users/sagen/resources-2/twext:5084-5093
/CalendarServer/branches/users/sagen/testing/twext:10827-10851,10853-10855
/CalendarServer/branches/users/wsanchez/transations/twext:5515-5593
/twext/branches/users/cdaboo/jobqueue-3/twext:13444-13471
/twext/branches/users/cdaboo/jobs/twext:12742-12780
/twext/branches/users/cdaboo/pod2pod-migration/twext:14416-14520
/twext/branches/users/sagen/recordtypes-2/twext:13659
/twext/branches/users/sagen/recordtypes/twext:13647-13658
Modified: twext/trunk/twext/enterprise/dal/record.py
===================================================================
--- twext/trunk/twext/enterprise/dal/record.py 2015-03-06 21:20:35 UTC (rev 14520)
+++ twext/trunk/twext/enterprise/dal/record.py 2015-03-06 21:39:59 UTC (rev 14521)
@@ -171,6 +171,18 @@
return r
+ def __hash__(self):
+ return hash(tuple([getattr(self, attr) for attr in self.__attrmap__.keys()]))
+
+
+ def __eq__(self, other):
+ if type(self) != type(other):
+ return False
+ attrs = dict([(attr, getattr(self, attr),) for attr in self.__attrmap__.keys()])
+ otherattrs = dict([(attr, getattr(other, attr),) for attr in other.__attrmap__.keys()])
+ return attrs == otherattrs
+
+
@classmethod
def fromTable(cls, table):
"""
@@ -198,7 +210,7 @@
words = columnName.lower().split("_")
def cap(word):
- if word.lower() == "id":
+ if word.lower() in ("id", "uid", "guid",):
return word.upper()
else:
return word.capitalize()
@@ -246,41 +258,50 @@
MyRecord.create(transaction, column1=1, column2=u"two")
"""
+ self = cls.make(**k)
+ yield self.insert(transaction)
+ returnValue(self)
+
+
+ @classmethod
+ def make(cls, **k):
+ """
+ Make a record without creating one in the database - this will not have an
+ associated L{transaction}. When the record is ready to be written to the database
+ use L{SerializeableRecord.insert} to add it. Before it gets written to the DB, the
+ attributes can be changed.
+ """
self = cls()
- colmap = {}
attrtocol = cls.__attrmap__
- needsCols = []
- needsAttrs = []
for attr in attrtocol:
col = attrtocol[attr]
if attr in k:
- setattr(self, attr, k[attr])
- colmap[col] = k.pop(attr)
+ value = k.pop(attr)
+ setattr(self, attr, value)
else:
if col.model.needsValue():
raise TypeError(
"required attribute {0!r} not passed"
.format(attr)
)
- else:
- needsCols.append(col)
- needsAttrs.append(attr)
if k:
raise TypeError("received unknown attribute{0}: {1}".format(
"s" if len(k) > 1 else "", ", ".join(sorted(k))
))
- result = yield (Insert(colmap, Return=needsCols if needsCols else None)
- .on(transaction))
- if needsCols:
- self._attributesFromRow(zip(needsAttrs, result[0]))
- self.transaction = transaction
+ return self
- returnValue(self)
+ def duplicate(self):
+ return self.make(**dict([(attr, getattr(self, attr),) for attr in self.__attrmap__.keys()]))
+
+ def isnew(self):
+ return self.transaction is None
+
+
def _attributesFromRow(self, attributeList):
"""
Take some data loaded from a row and apply it to this instance,
@@ -296,6 +317,46 @@
setattr(self, setAttribute, setValue)
+ @inlineCallbacks
+ def insert(self, transaction):
+ """
+ Insert a new a row for an existing record that was not initially created in the database.
+ """
+
+ # Cannot do this if a transaction has already been assigned because that means
+ # the record already exists in the DB.
+
+ if self.transaction is not None:
+ raise ReadOnly(self.__class__.__name__, "Cannot insert")
+
+ colmap = {}
+ attrtocol = self.__attrmap__
+ needsCols = []
+ needsAttrs = []
+
+ for attr in attrtocol:
+ col = attrtocol[attr]
+ v = getattr(self, attr)
+ if not isinstance(v, ColumnSyntax):
+ colmap[col] = v
+ else:
+ if col.model.needsValue():
+ raise TypeError(
+ "required attribute {0!r} not passed"
+ .format(attr)
+ )
+ else:
+ needsCols.append(col)
+ needsAttrs.append(attr)
+
+ result = yield (Insert(colmap, Return=needsCols if needsCols else None)
+ .on(transaction))
+ if needsCols:
+ self._attributesFromRow(zip(needsAttrs, result[0]))
+
+ self.transaction = transaction
+
+
def delete(self):
"""
Delete this row from the database.
@@ -411,7 +472,7 @@
@classmethod
- def query(cls, transaction, expr, order=None, ascending=True, group=None, forUpdate=False, noWait=False, limit=None):
+ def query(cls, transaction, expr, order=None, group=None, limit=None, forUpdate=False, noWait=False, ascending=True, distinct=False):
"""
Query the table that corresponds to C{cls}, and return instances of
C{cls} corresponding to the rows that are returned from that table.
@@ -435,30 +496,87 @@
@param noWait: include NOWAIT with the FOR UPDATE
@type noWait: L{bool}
"""
+ return cls._rowsFromQuery(
+ transaction,
+ cls.queryExpr(
+ expr,
+ order=order,
+ group=group,
+ limit=limit,
+ forUpdate=forUpdate,
+ noWait=noWait,
+ ascending=ascending,
+ distinct=distinct,
+ ),
+ None
+ )
+
+
+ @classmethod
+ def queryExpr(cls, expr, attributes=None, order=None, group=None, limit=None, forUpdate=False, noWait=False, ascending=True, distinct=False):
+ """
+ Query expression that corresponds to C{cls}. Used in cases where a sub-select
+ on this record's table is needed.
+
+ @param expr: An L{ExpressionSyntax} that constraints the results of the
+ query. This is most easily produced by accessing attributes on the
+ class; for example, C{MyRecordType.query((MyRecordType.col1 >
+ MyRecordType.col2).And(MyRecordType.col3 == 7))}
+
+ @param order: A L{ColumnSyntax} to order the resulting record objects
+ by.
+
+ @param ascending: A boolean; if C{order} is not C{None}, whether to
+ sort in ascending or descending order.
+
+ @param group: a L{ColumnSyntax} to group the resulting record objects
+ by.
+
+ @param forUpdate: do a SELECT ... FOR UPDATE
+ @type forUpdate: L{bool}
+ @param noWait: include NOWAIT with the FOR UPDATE
+ @type noWait: L{bool}
+ """
kw = {}
if order is not None:
kw.update(OrderBy=order, Ascending=ascending)
if group is not None:
kw.update(GroupBy=group)
+ if limit is not None:
+ kw.update(Limit=limit)
if forUpdate:
kw.update(ForUpdate=True)
if noWait:
kw.update(NoWait=True)
- if limit is not None:
- kw.update(Limit=limit)
- return cls._rowsFromQuery(
- transaction,
- Select(
- list(cls.table),
- From=cls.table,
- Where=expr,
- **kw
- ),
- None
+ if distinct:
+ kw.update(Distinct=True)
+ if attributes is None:
+ attributes = list(cls.table)
+ return Select(
+ attributes,
+ From=cls.table,
+ Where=expr,
+ **kw
)
@classmethod
+ def querysimple(cls, transaction, **kw):
+ """
+ Match all rows matching the specified attribute/values from the table that corresponds to C{cls}.
+ All attributes are logically AND'ed.
+ """
+ where = None
+ for k, v in kw.iteritems():
+ subexpr = (cls.__attrmap__[k] == v)
+ if where is None:
+ where = subexpr
+ else:
+ where = where.And(subexpr)
+ return cls.query(transaction, where)
+
+
+ @classmethod
def all(cls, transaction):
"""
Load all rows from the table that corresponds to C{cls} and return
@@ -484,17 +602,34 @@
@classmethod
- def deletesome(cls, transaction, where):
+ def deletesome(cls, transaction, where, returnCols=None):
"""
Delete all rows matching the where expression from the table that corresponds to C{cls}.
"""
return Delete(
From=cls.table,
Where=where,
+ Return=returnCols,
).on(transaction)
@classmethod
+ def deletesimple(cls, transaction, **kw):
+ """
+ Delete all rows matching the specified attribute/values from the table that corresponds to C{cls}.
+ All attributes are logically AND'ed.
+ """
+ where = None
+ for k, v in kw.iteritems():
+ subexpr = (cls.__attrmap__[k] == v)
+ if where is None:
+ where = subexpr
+ else:
+ where = where.And(subexpr)
+ return cls.deletesome(transaction, where)
+
+
+ @classmethod
@inlineCallbacks
def _rowsFromQuery(cls, transaction, qry, rozrc):
"""
@@ -521,3 +656,61 @@
self.transaction = transaction
selves.append(self)
returnValue(selves)
+
+
+
+class SerializableRecord(Record):
+ """
+ An L{Record} that serializes/deserializes its attributes for a text-based
+ transport (e.g., JSON-over-HTTP) to allow records to be transferred from
+ one system to another (with potentially mismatched schemas).
+ """
+
+ def serialize(self):
+ """
+ Create an L{dict} of each attribute with L{str} values for each attribute
+ value. Sub-classes may need to override this to specialize certain value
+ conversions.
+
+ @return: mapping of attribute to string values
+ @rtype: L{dict} of L{str}:L{str}
+ """
+
+ # Certain values have to be mapped to str
+ result = {}
+ for attr in self.__attrmap__:
+ value = getattr(self, attr)
+ col = self.__attrmap__[attr]
+ if col.model.type.name == "timestamp" and value is not None:
+ value = str(value)
+ result[attr] = value
+ return result
+
+
+ @classmethod
+ def deserialize(cls, attrmap):
+ """
+ Given an L{dict} mapping attributes to values, create an L{Record} with
+ the specified values. Sub-classes may need to override this to handle special
+ values that need to be converted to specific types. They also need to override
+ this to handle possible schema mismatches (attributes no longer used, new
+ attributes not present in the map).
+
+ @param attrmap: serialized representation of a record
+ @type attrmap: L{dict} of L{str}:L{str}
+
+ @return: a newly created, but not inserted, record
+ @rtype: L{SerializableRecord}
+ """
+
+ # Certain values have to be mapped to non-str types
+ mapped = {}
+ for attr in attrmap:
+ value = attrmap[attr]
+ col = cls.__attrmap__[attr]
+ if col.model.type.name == "timestamp" and value is not None:
+ value = parseSQLTimestamp(value)
+ mapped[attr] = value
+
+ record = cls.make(**mapped)
+ return record
Modified: twext/trunk/twext/enterprise/dal/syntax.py
===================================================================
--- twext/trunk/twext/enterprise/dal/syntax.py 2015-03-06 21:20:35 UTC (rev 14520)
+++ twext/trunk/twext/enterprise/dal/syntax.py 2015-03-06 21:39:59 UTC (rev 14521)
@@ -439,6 +439,8 @@
"{} expression needs an explicit count of parameters".format(op.upper())
)
return CompoundComparison(self, op, Constant(other))
+ elif isinstance(other, set) or isinstance(other, frozenset) or isinstance(other, list) or isinstance(other, tuple):
+ return CompoundComparison(self, op, Constant(other))
else:
# Can't be Select.__contains__ because __contains__ gets
# __nonzero__ called on its result by the "in" syntax.
@@ -548,6 +550,13 @@
for counter in range(self.value.count)
]).subSQL(queryGenerator, allTables)
)
+ elif isinstance(self.value, set) or isinstance(self.value, frozenset) or isinstance(self.value, list) or isinstance(self.value, tuple):
+ return _inParens(
+ _CommaList([
+ SQLFragment(queryGenerator.placeholder.placeholder(), [value])
+ for value in self.value
+ ]).subSQL(queryGenerator, allTables)
+ )
else:
return SQLFragment(
queryGenerator.placeholder.placeholder(), [self.value]
Modified: twext/trunk/twext/enterprise/dal/test/test_record.py
===================================================================
--- twext/trunk/twext/enterprise/dal/test/test_record.py 2015-03-06 21:20:35 UTC (rev 14520)
+++ twext/trunk/twext/enterprise/dal/test/test_record.py 2015-03-06 21:39:59 UTC (rev 14521)
@@ -24,8 +24,8 @@
from twisted.trial.unittest import TestCase, SkipTest
from twext.enterprise.dal.record import (
- Record, fromTable, ReadOnly, NoSuchRecord
-)
+ Record, fromTable, ReadOnly, NoSuchRecord,
+ SerializableRecord)
from twext.enterprise.dal.test.test_parseschema import SchemaTestHelper
from twext.enterprise.dal.syntax import SchemaSyntax
from twext.enterprise.fixtures import buildConnectionPool
@@ -67,6 +67,13 @@
+class TestSerializeRecord(SerializableRecord, Alpha):
+ """
+ A sample test serializable record with default values specified.
+ """
+
+
+
class TestAutoRecord(Record, Delta):
"""
A sample test record with default values specified.
@@ -300,6 +307,44 @@
@inlineCallbacks
+ def test_querySimple(self):
+ """
+ L{Record.querysimple} will allow you to query for a record by its class
+ attributes as columns.
+ """
+ txn = self.pool.connection()
+ for beta, gamma in [(123, u"one"), (234, u"two"), (345, u"three"),
+ (356, u"three"), (456, u"four")]:
+ yield txn.execSQL("insert into ALPHA values (:1, :2)",
+ [beta, gamma])
+ records = yield TestRecord.querysimple(txn, gamma=u"three")
+ self.assertEqual(len(records), 2)
+ records.sort(key=lambda x: x.beta)
+ self.assertEqual(records[0].beta, 345)
+ self.assertEqual(records[1].beta, 356)
+
+
+ @inlineCallbacks
+ def test_eq(self):
+ """
+ L{Record.__eq__} works.
+ """
+ txn = self.pool.connection()
+ data = [(123, u"one"), (456, u"four"), (345, u"three"),
+ (234, u"two"), (356, u"three")]
+ for beta, gamma in data:
+ yield txn.execSQL("insert into ALPHA values (:1, :2)",
+ [beta, gamma])
+
+ one = yield TestRecord.load(txn, 123)
+ one_copy = yield TestRecord.load(txn, 123)
+ two = yield TestRecord.load(txn, 234)
+
+ self.assertTrue(one == one_copy)
+ self.assertFalse(one == two)
+
+
+ @inlineCallbacks
def test_all(self):
"""
L{Record.all} will return all instances of the record, sorted by
@@ -335,6 +380,48 @@
@inlineCallbacks
+ def test_deletesome(self):
+ """
+ L{Record.deletesome} will delete all instances of the matching records.
+ """
+ txn = self.pool.connection()
+ data = [(123, u"one"), (456, u"four"), (345, u"three"),
+ (234, u"two"), (356, u"three")]
+ for beta, gamma in data:
+ yield txn.execSQL("insert into ALPHA values (:1, :2)",
+ [beta, gamma])
+
+ yield TestRecord.deletesome(txn, TestRecord.gamma == u"three")
+ all = yield TestRecord.all(txn)
+ self.assertEqual(set([record.beta for record in all]), set((123, 456, 234,)))
+
+ yield TestRecord.deletesome(txn, (TestRecord.gamma == u"one").Or(TestRecord.gamma == u"two"))
+ all = yield TestRecord.all(txn)
+ self.assertEqual(set([record.beta for record in all]), set((456,)))
+
+
+ @inlineCallbacks
+ def test_deletesimple(self):
+ """
+ L{Record.deletesimple} will delete all instances of the matching records.
+ """
+ txn = self.pool.connection()
+ data = [(123, u"one"), (456, u"four"), (345, u"three"),
+ (234, u"two"), (356, u"three")]
+ for beta, gamma in data:
+ yield txn.execSQL("insert into ALPHA values (:1, :2)",
+ [beta, gamma])
+
+ yield TestRecord.deletesimple(txn, gamma=u"three")
+ all = yield TestRecord.all(txn)
+ self.assertEqual(set([record.beta for record in all]), set((123, 456, 234,)))
+
+ yield TestRecord.deletesimple(txn, beta=123, gamma=u"one")
+ all = yield TestRecord.all(txn)
+ self.assertEqual(set([record.beta for record in all]), set((456, 234)))
+
+
+ @inlineCallbacks
def test_repr(self):
"""
The C{repr} of a L{Record} presents all its values.
@@ -458,3 +545,59 @@
rec = yield TestRecord.load(txn, 234)
result = yield rec.trylock()
self.assertTrue(result)
+
+
+ @inlineCallbacks
+ def test_serialize(self):
+ """
+ A L{SerializableRecord} may be serialized.
+ """
+ txn = self.pool.connection()
+ for beta, gamma in [
+ (123, u"one"),
+ (234, u"two"),
+ (345, u"three"),
+ (356, u"three"),
+ (456, u"four"),
+ ]:
+ yield txn.execSQL(
+ "insert into ALPHA values (:1, :2)", [beta, gamma]
+ )
+
+ rec = yield TestSerializeRecord.load(txn, 234)
+ result = rec.serialize()
+ self.assertEqual(result, {"beta": 234, "gamma": u"two"})
+
+
+ @inlineCallbacks
+ def test_deserialize(self):
+ """
+ A L{SerializableRecord} may be deserialized.
+ """
+ txn = self.pool.connection()
+
+ rec = yield TestSerializeRecord.deserialize({"beta": 234, "gamma": u"two"})
+ yield rec.insert(txn)
+ yield txn.commit()
+
+ txn = self.pool.connection()
+ rec = yield TestSerializeRecord.query(txn, TestSerializeRecord.beta == 234)
+ self.assertEqual(len(rec), 1)
+ self.assertEqual(rec[0].gamma, u"two")
+ yield txn.commit()
+
+ # Check that attributes can be changed prior to insert, and not after
+ txn = self.pool.connection()
+ rec = yield TestSerializeRecord.deserialize({"beta": 456, "gamma": u"one"})
+ rec.gamma = u"four"
+ yield rec.insert(txn)
+ def _raise():
+ rec.gamma = u"five"
+ self.assertRaises(ReadOnly, _raise)
+ yield txn.commit()
+
+ txn = self.pool.connection()
+ rec = yield TestSerializeRecord.query(txn, TestSerializeRecord.beta == 456)
+ self.assertEqual(len(rec), 1)
+ self.assertEqual(rec[0].gamma, u"four")
+ yield txn.commit()
Modified: twext/trunk/twext/enterprise/dal/test/test_sqlsyntax.py
===================================================================
--- twext/trunk/twext/enterprise/dal/test/test_sqlsyntax.py 2015-03-06 21:20:35 UTC (rev 14520)
+++ twext/trunk/twext/enterprise/dal/test/test_sqlsyntax.py 2015-03-06 21:39:59 UTC (rev 14521)
@@ -1006,6 +1006,90 @@
)
+ def test_inIterable(self):
+ """
+ L{ColumnSyntax.In} returns a sub-expression using the SQL C{in} syntax
+ with parameter list.
+ """
+ # One item with IN only
+ items = set(("A",))
+ for items in (set(items), list(items), tuple(items),):
+ self.assertEquals(
+ Select(
+ From=self.schema.FOO,
+ Where=self.schema.FOO.BAR.In(items),
+ ).toSQL().bind(),
+ SQLFragment("select * from FOO where BAR in (?)", ["A"])
+ )
+
+ # Two items with IN only
+ items = set(("A", "B"))
+ for items in (set(items), list(items), tuple(items),):
+ self.assertEquals(
+ Select(
+ From=self.schema.FOO,
+ Where=self.schema.FOO.BAR.In(items),
+ ).toSQL().bind(names=items),
+ SQLFragment(
+ "select * from FOO where BAR in (?, ?)", ["A", "B"]
+ )
+ )
+
+ # Two items with preceding AND
+ self.assertEquals(
+ Select(
+ From=self.schema.FOO,
+ Where=(
+ (
+ self.schema.FOO.BAZ == Parameter("P1")
+ ).And(
+ self.schema.FOO.BAR.In(items)
+ )
+ )
+ ).toSQL().bind(P1="P1"),
+ SQLFragment(
+ "select * from FOO where BAZ = ? and BAR in (?, ?)",
+ ["P1", "A", "B"]
+ ),
+ )
+
+ # Two items with following AND
+ self.assertEquals(
+ Select(
+ From=self.schema.FOO,
+ Where=(
+ (
+ self.schema.FOO.BAR.In(items)
+ ).And(
+ self.schema.FOO.BAZ == Parameter("P2")
+ )
+ )
+ ).toSQL().bind(P2="P2"),
+ SQLFragment(
+ "select * from FOO where BAR in (?, ?) and BAZ = ?",
+ ["A", "B", "P2"]
+ ),
+ )
+
+ # Two items with preceding OR and following AND
+ self.assertEquals(
+ Select(
+ From=self.schema.FOO,
+ Where=((
+ self.schema.FOO.BAZ == Parameter("P1")
+ ).Or(
+ self.schema.FOO.BAR.In(items).And(
+ self.schema.FOO.BAZ == Parameter("P2")
+ )
+ ))
+ ).toSQL().bind(P1="P1", P2="P2"),
+ SQLFragment(
+ "select * from FOO where BAZ = ? or BAR in (?, ?) and BAZ = ?",
+ ["P1", "A", "B", "P2"]
+ ),
+ )
+
+
def test_max(self):
"""
L{Max}C{(column)} produces an object in the C{columns} clause that
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20150306/abaed1a9/attachment-0001.html>
More information about the calendarserver-changes
mailing list