�
Ϫ�fc\ � �� � d Z ddlmZ ddlmZmZmZ ddlmZ ddl m
Z
ddl m Z ddl
mZmZ dZ G d � d
e� Z G d � d e� Z G d
� de� Z G d� d� Z G d� dee
� Zg d�Zy)ap
Memcache client protocol. Memcached is a caching server, storing data in the
form of pairs key/value, and memcache is the protocol to talk with it.
To connect to a server, create a factory for L{MemCacheProtocol}::
from twisted.internet import reactor, protocol
from twisted.protocols.memcache import MemCacheProtocol, DEFAULT_PORT
d = protocol.ClientCreator(reactor, MemCacheProtocol
).connectTCP("localhost", DEFAULT_PORT)
def doSomething(proto):
# Here you call the memcache operations
return proto.set("mykey", "a lot of data")
d.addCallback(doSomething)
reactor.run()
All the operations of the memcache protocol are present, but
L{MemCacheProtocol.set} and L{MemCacheProtocol.get} are the more important.
See U{http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt} for
more information about the protocol.
� )�deque)�Deferred� TimeoutError�fail)� LineReceiver)� TimeoutMixin)�log)� nativeString�
networkStringi�+ c � � e Zd ZdZy)�
NoSuchCommandzA
Exception raised when a non existent command is called.
N��__name__�
__module__� __qualname__�__doc__� � �</usr/lib/python3/dist-packages/twisted/protocols/memcache.pyr
r
( � � �r r
c � � e Zd ZdZy)� ClientErrorz1
Error caused by an invalid client call.
Nr r r r r r . r r r c � � e Zd ZdZy)� ServerErrorz*
Problem happening on the server.
Nr r r r r r 4 r r r c �" � e Zd ZdZd� Zd� Zd� Zy)�Commanda8
Wrap a client action into an object, that holds the values used in the
protocol.
@ivar _deferred: the L{Deferred} object that will be fired when the result
arrives.
@type _deferred: L{Deferred}
@ivar command: name of the command sent to the server.
@type command: L{bytes}
c �z � || _ t � | _ |j � D ] \ }}t | ||� � y)z�
Create a command.
@param command: the name of the command.
@type command: L{bytes}
@param kwargs: this values will be stored as attributes of the object
for future use
N)�commandr � _deferred�items�setattr)�selfr �kwargs�k�vs r �__init__zCommand.__init__G s9 � � �� �!�����L�L�N� �D�A�q� �D�!�Q� � r c �: � | j j |� y)zB
Shortcut method to fire the underlying deferred.
N)r �callback)r" �values r �successzCommand.successV s � �
��� � ��&r c �: � | j j |� y)z5
Make the underlying deferred fails.
N)r �errback)r" �errors r r z Command.fail\ s � �
��� � �u�%r N)r r r r r& r* r r r r r r : s � �
�
� '� &r r c �
� e Zd ZdZdZdZd*d�Zd� Zd� Zd� Z d� Z
d � Z d
� Z d � Z
d � Zd
� Zd� Zd� Zd� Zd� Zd� Zd� Zd� Zd� Zd� Zd� Zd+d�Zd+d�Zd� Z d,d�Z d,d �Z d,d �Z d,d �Z d � Z!d � Z"d!� Z#d-d"�Z$d-d#�Z%d$� Z&d.d&�Z'd'� Z(d(� Z)d)� Z*y%)/�MemCacheProtocola1
MemCache protocol: connect to a memcached server to store/retrieve values.
@ivar persistentTimeOut: the timeout period used to wait for a response.
@type persistentTimeOut: L{int}
@ivar _current: current list of requests waiting for an answer from the
server.
@type _current: L{deque} of L{Command}
@ivar _lenExpected: amount of data expected in raw mode, when reading for
a value.
@type _lenExpected: L{int}
@ivar _getBuffer: current buffer of data, used to store temporary data
when reading in raw mode.
@type _getBuffer: L{list}
@ivar _bufferLength: the total amount of bytes in C{_getBuffer}.
@type _bufferLength: L{int}
@ivar _disconnected: indicate if the connectionLost has been called or not.
@type _disconnected: L{bool}
� Fc �h � t � | _ d| _ d| _ d| _ |x| _ | _ y)z�
Create the protocol.
@param timeOut: the timeout to wait before detecting that the
connection is dead and close it. It's expressed in seconds.
@type timeOut: L{int}
N)r �_current� _lenExpected�
_getBuffer�
_bufferLength�persistentTimeOut�timeOut)r" r7 s r r& zMemCacheProtocol.__init__� s3 � � ���
� ��� ��� !���07�7�� �� r c � � | j r9| j j � }|j |� | j r�8yy)zW
Cancel all the outstanding commands, making them fail with C{reason}.
N)r2 �popleftr )r" �reason�cmds r �_cancelCommandsz MemCacheProtocol._cancelCommands� s2 � � �m�m��-�-�'�'�)�C� �H�H�V� � �m�mr c �l � | j t d� � | j j � y)z:
Close the connection in case of timeout.
zConnection timeoutN)r< r � transport�loseConnection�r" s r �timeoutConnectionz"MemCacheProtocol.timeoutConnection� s( � �
� � �\�*>� ?�@� ���%�%�'r c �` � d| _ | j |� t j | |� y)z9
Cause any outstanding commands to fail.
TN)�
_disconnectedr< r �connectionLost)r" r: s r rD z MemCacheProtocol.connectionLost� s* � � "��� � � �V�$��#�#�D�&�1r c �~ � | j s| j | j � t j | |� y)zA
Override sendLine to add a timeout to response.
N)r2 �
setTimeoutr6 r �sendLine)r" �lines r rG zMemCacheProtocol.sendLine� s- � � �}�}� �O�O�D� 2� 2� 3�� � �d�D�)r c �N � | j � | j j |� | xj t |� z
c_ | j | j
dz k\ r�dj
| j � }|d| j
}|| j
dz d }|}d| _ d| _ d| _ | j d }|j r9|j |j \ }}|||f|j |j < n||_ | j |� yy)z)
Collect data for a get.
� r Nr )
� resetTimeoutr4 �appendr5 �lenr3 �joinr2 �multiple�values�
currentKeyr) � setLineMode)r" �data�buf�rem�valr; �flags�cass r �rawDataReceivedz MemCacheProtocol.rawDataReceived� s � �
���� ��� � �t�$� ���c�$�i�'�� � � ��!2�!2�Q�!6� 6��8�8�D�O�O�,�D��*��*�*�+�C��t�(�(�1�,�.�/�C��C� $�D� � "�D�O�!%�D� ��-�-��"�C��|�|� �Z�Z���� 7�
��s�.3�S�#�->��
�
�3�>�>�*� �� � � � �S� !� 7r c �V � | j j � j d� y)z?
Manage a success response to a set operation.
TN�r2 r9 r* r@ s r �
cmd_STOREDzMemCacheProtocol.cmd_STORED� � � �
�
�
� � � �'�'��-r c �V � | j j � j d� y)z�
Manage a specific 'not stored' response to a set operation: this is not
an error, but some condition wasn't met.
FNr[ r@ s r �cmd_NOT_STOREDz MemCacheProtocol.cmd_NOT_STORED� s � �
�
�
� � � �'�'��.r c �� � | j j � }|j dk( ry|j rE|j j � D ��ci c] \ }}||ddd� �� }}}|j
|� y|j
|j |j f� y|j dk( r[|j r |j
|j � y|j
|j |j |j f� y|j dk( r |j
|j � yt dj t |j � � � �c c}}w )zB
This the end token to a get or a stat operation.
� getNrJ � gets� statsz%Unexpected END response to {} command)
r2 r9 r rO rP r r* rW r) rX � RuntimeError�formatr
)r" r; �keyrV rP s r �cmd_ENDzMemCacheProtocol.cmd_END� s � � �m�m�#�#�%�� �;�;�&� ��|�|�8;�
�
�8H�8H�8J�K�H�C��#�s�3�Q�3�x�-�K��K�� � �F�#�� � �S�Y�Y�� � � 2�3�
�[�[�G�
#��|�|�� � �C�J�J�'�� � �S�Y�Y������� ;�<�
�[�[�H�
$� �K�K��
�
� #� �7�>�>� ����-���
�� Ls �E%c �V � | j j � j d� y)z=
Manage error response for incr/decr/delete.
FNr[ r@ s r �
cmd_NOT_FOUNDz MemCacheProtocol.cmd_NOT_FOUND� � � �
�
�
� � � �'�'��.r c �� � | j d }|j dk( r|j � \ }}}d}n|j � \ }}}}t |� | _ g | _ d| _ |j r;||j vr t d� �||_
t |� |g|j |< n1|j |k7 r t d� �t |� |_
||_ | j � y)z:
Prepare the reading a value after a get.
r ra r zUnexpected commands answer.N)r2 r �split�intr3 r4 r5 rO �keysrd rQ rP rf rW rX �
setRawMode)r" rH r; rf rW �lengthrX s r � cmd_VALUEzMemCacheProtocol.cmd_VALUE� s� � � �m�m�A� �� �;�;�&� �!%��� � �C����C�&*�j�j�l� #�C���� ��K��� ��� ��� �<�<��#�(�(�"�"�#@�A�A� �C�N� "�5�z�3� /�C�J�J�s�O��w�w�#�~�"�#@�A�A��E�
�C�I��C�G� ���r c �j � | j d }|j dd� \ }}||j |< y)z-
Reception of one stat line.
r � � N)r2 rl rP )r" rH r; rf rV s r �cmd_STATzMemCacheProtocol.cmd_STAT s3 � � �m�m�A� ���:�:�d�A�&���S� ��
�
�3�r c �V � | j j � j |� y)z%
Read version token.
Nr[ )r" � versionDatas r � cmd_VERSIONz MemCacheProtocol.cmd_VERSION s � �
�
�
� � � �'�'� �4r c � � t j d� | j j � }|j t � � y)z7
A non-existent command has been sent.
zNon-existent command sent.N)r �errr2 r9 r r
)r" r; s r � cmd_ERRORzMemCacheProtocol.cmd_ERROR s1 � � ���,�-��m�m�#�#�%�� ��� � �!r c � � t |� }t j d|z � | j j � }|j t
|� � y)z0
An invalid input as been sent.
zInvalid input: N)�reprr rz r2 r9 r r �r" �errTextr; s r �cmd_CLIENT_ERRORz!MemCacheProtocol.cmd_CLIENT_ERROR sB � � �w�-�� ���!�G�+�,��m�m�#�#�%�� ����W�%�&r c � � t |� }t j d|z � | j j � }|j t
|� � y)z4
An error has happened server-side.
zServer error: N)r} r rz r2 r9 r r r~ s r �cmd_SERVER_ERRORz!MemCacheProtocol.cmd_SERVER_ERROR( sB � � �w�-�� ��� �7�*�+��m�m�#�#�%�� ����W�%�&r c �V � | j j � j d� y)z>
A delete command has completed successfully.
TNr[ r@ s r � cmd_DELETEDz MemCacheProtocol.cmd_DELETED1 r] r c �V � | j j � j d� y)z6
The last command has been completed.
TNr[ r@ s r �cmd_OKzMemCacheProtocol.cmd_OK7 r] r c �V � | j j � j d� y)z5
A C{checkAndSet} update has failed.
FNr[ r@ s r �
cmd_EXISTSzMemCacheProtocol.cmd_EXISTS= rj r c �� � | j � |j dd� d }t | dt |� z d� }|�+|j dd� dd }|r ||d � ns |� nk|j dd� }t | dt |� z d� }|� |� n6| j
j
� }t |� }|j |� | j
s| j d� yy)z8
Receive line commands from the server.
rs rt r �cmd_N� _)
rK rl �getattrr
�replacer2 r9 rm r* rF )r" rH �tokenr; �argsrV s r � lineReceivedz MemCacheProtocol.lineReceivedC s� � �
�����
�
�4��#�A�&���d�F�\�%�%8� 8�$�?�� �?��:�:�d�A�&�q�r�*�D���D��G� ��� �<�<��d�+�D��$�� �d�);� ;�T�B�C����� �m�m�+�+�-���$�i��� � �C� ��}�}� �O�O�D� !� r c �( � | j d||� S )a�
Increment the value of C{key} by given value (default to 1).
C{key} must be consistent with an int. Return the new value.
@param key: the key to modify.
@type key: L{bytes}
@param val: the value to increment.
@type val: L{int}
@return: a deferred with will be called back with the new value
associated with the key (after the increment).
@rtype: L{Deferred}
s incr�� _incrdecr�r" rf rV s r � incrementzMemCacheProtocol.increment` s � � �~�~�g�s�C�0�0r c �( � | j d||� S )a�
Decrement the value of C{key} by given value (default to 1).
C{key} must be consistent with an int. Return the new value, coerced to
0 if negative.
@param key: the key to modify.
@type key: L{bytes}
@param val: the value to decrement.
@type val: L{int}
@return: a deferred with will be called back with the new value
associated with the key (after the decrement).
@rtype: L{Deferred}
s decrr� r� s r � decrementzMemCacheProtocol.decrementq s � � �~�~�g�s�C�0�0r c �� � | j rt t d� � S t |t � s!t t dt
|� � d�� � S t |� | j kD rt t d� � S dj ||dt |� fz g� }| j |� t ||�� }| j j |� |j S )z1
Internal wrapper for incr/decr.
�
not connected�Invalid type for key: �, expecting bytes� Key too longrs s %d�rf )rC r rd �
isinstance�bytesr �typerM �MAX_KEY_LENGTHrN rm rG r r2 rL r )r" r; rf rV �fullcmd�cmdObjs r r� zMemCacheProtocol._incrdecr� s� � � � � �� �_�5�6� 6��#�u�%��� 4�T�#�Y�K�?P� Q�R��
� �s�8�d�)�)� )�� �N�3�4� 4��)�)�S�#�u��C��{�':� ;�<�� �
�
�g� ���#�&�� �
�
� � �V�$�� � � r c �. � | j d||||d� S )a�
Replace the given C{key}. It must already exist in the server.
@param key: the key to replace.
@type key: L{bytes}
@param val: the new value associated with the key.
@type val: L{bytes}
@param flags: the flags to store with the key.
@type flags: L{int}
@param expireTime: if different from 0, the relative time in seconds
when the key will be deleted from the store.
@type expireTime: L{int}
@return: a deferred that will fire with C{True} if the operation has
succeeded, and C{False} with the key didn't previously exist.
@rtype: L{Deferred}
s replacer ��_set�r" rf rV rW �
expireTimes r r� zMemCacheProtocol.replace� s � �* �y�y��S�#�u�j�#�F�Fr c �. � | j d||||d� S )a�
Add the given C{key}. It must not exist in the server.
@param key: the key to add.
@type key: L{bytes}
@param val: the value associated with the key.
@type val: L{bytes}
@param flags: the flags to store with the key.
@type flags: L{int}
@param expireTime: if different from 0, the relative time in seconds
when the key will be deleted from the store.
@type expireTime: L{int}
@return: a deferred that will fire with C{True} if the operation has
succeeded, and C{False} with the key already exists.
@rtype: L{Deferred}
s addr r� r� s r �addzMemCacheProtocol.add� � � �* �y�y���c�5�*�c�B�Br c �. � | j d||||d� S )a9
Set the given C{key}.
@param key: the key to set.
@type key: L{bytes}
@param val: the value associated with the key.
@type val: L{bytes}
@param flags: the flags to store with the key.
@type flags: L{int}
@param expireTime: if different from 0, the relative time in seconds
when the key will be deleted from the store.
@type expireTime: L{int}
@return: a deferred that will fire with C{True} if the operation has
succeeded.
@rtype: L{Deferred}
s setr r� r� s r �setzMemCacheProtocol.set� r� r c �. � | j d|||||� S )am
Change the content of C{key} only if the C{cas} value matches the
current one associated with the key. Use this to store a value which
hasn't been modified since last time you fetched it.
@param key: The key to set.
@type key: L{bytes}
@param val: The value associated with the key.
@type val: L{bytes}
@param cas: Unique 64-bit value returned by previous call of C{get}.
@type cas: L{bytes}
@param flags: The flags to store with the key.
@type flags: L{int}
@param expireTime: If different from 0, the relative time in seconds
when the key will be deleted from the store.
@type expireTime: L{int}
@return: A deferred that will fire with C{True} if the operation has
succeeded, C{False} otherwise.
@rtype: L{Deferred}
s casr� )r" rf rV rX rW r� s r � checkAndSetz MemCacheProtocol.checkAndSet� s � �4 �y�y���c�5�*�c�B�Br c
�~ � | j rt t d� � S t |t � s!t t dt
|� � d�� � S t |� | j kD rt t d� � S t |t � s!t t dt
|� � d�� � S |rd|z }t |� }dj ||t d|||fz � g� |z }| j |� | j |� t ||||�� } | j j | � | j S ) z6
Internal wrapper for setting values.
r� r� r� r� zInvalid type for value: rs z%d %d %d)rf rW rp )rC r rd r� r� r r� rM r� rN r rG r r2 rL r )
r" r; rf rV rW r� rX rp r� r� s
r r� zMemCacheProtocol._set� s2 � � � � �� �_�5�6� 6��#�u�%��� 4�T�#�Y�K�?P� Q�R��
� �s�8�d�)�)� )�� �N�3�4� 4��#�u�%��� 6�t�C�y�k�AR� S�T��
� ���*�C��S��� �I�I��c�=��u�j�&�6Q�)Q�R�S�
� �
� �
�
�
�g� � �
�
�c����#�U�6�B�� �
�
� � �V�$�� � � r c �. � | j d||ddd� S )a�
Append given data to the value of an existing key.
@param key: The key to modify.
@type key: L{bytes}
@param val: The value to append to the current value associated with
the key.
@type val: L{bytes}
@return: A deferred that will fire with C{True} if the operation has
succeeded, C{False} otherwise.
@rtype: L{Deferred}
s appendr r r� r� s r rL zMemCacheProtocol.append s � � �y�y��C��a��C�8�8r c �. � | j d||ddd� S )a�
Prepend given data to the value of an existing key.
@param key: The key to modify.
@type key: L{bytes}
@param val: The value to prepend to the current value associated with
the key.
@type val: L{bytes}
@return: A deferred that will fire with C{True} if the operation has
succeeded, C{False} otherwise.
@rtype: L{Deferred}
s prependr r r� r� s r �prependzMemCacheProtocol.prepend' s � � �y�y��S�#�q�!�S�9�9r c �* � | j |g|d� S )a�
Get the given C{key}. It doesn't support multiple keys. If
C{withIdentifier} is set to C{True}, the command issued is a C{gets},
that will return the current identifier associated with the value. This
identifier has to be used when issuing C{checkAndSet} update later,
using the corresponding method.
@param key: The key to retrieve.
@type key: L{bytes}
@param withIdentifier: If set to C{True}, retrieve the current
identifier along with the value and the flags.
@type withIdentifier: L{bool}
@return: A deferred that will fire with the tuple (flags, value) if
C{withIdentifier} is C{False}, or (flags, cas identifier, value)
if C{True}. If the server indicates there is no value
associated with C{key}, the returned value will be L{None} and
the returned flags will be C{0}.
@rtype: L{Deferred}
F��_get)r" rf �withIdentifiers r �getzMemCacheProtocol.get9 s � �, �y�y�#����6�6r c �( � | j ||d� S )a
Get the given list of C{keys}. If C{withIdentifier} is set to C{True},
the command issued is a C{gets}, that will return the identifiers
associated with each values. This identifier has to be used when
issuing C{checkAndSet} update later, using the corresponding method.
@param keys: The keys to retrieve.
@type keys: L{list} of L{bytes}
@param withIdentifier: If set to C{True}, retrieve the identifiers
along with the values and the flags.
@type withIdentifier: L{bool}
@return: A deferred that will fire with a dictionary with the elements
of C{keys} as keys and the tuples (flags, value) as values if
C{withIdentifier} is C{False}, or (flags, cas identifier, value) if
C{True}. If the server indicates there is no value associated with
C{key}, the returned values will be L{None} and the returned flags
will be C{0}.
@rtype: L{Deferred}
@since: 9.0
Tr� )r" rn r� s r � getMultiplez MemCacheProtocol.getMultipleQ s � �0 �y�y��~�t�4�4r c �F � t |� }| j rt t d� � S |D ]c }t |t
� s#t t
dt |� � d�� � c S t |� | j kD s�Ot t
d� � c S |rd}nd}dj |g|z � }| j |� |r |D �ci c] }|d�� }}t |||d �
� }nt ||d d d d
d�� }| j j |� |j S c c}w )z>
Helper method for C{get} and C{getMultiple}.
r� r� r� r� rb ra rs )r r NT)rn rP rO r Nr F)rf r) rW rX rO )�listrC r rd r� r� r r� rM r� rN rG r r2 rL r ) r" rn r� rO rf r; r� rP r� s r r� zMemCacheProtocol._getk s! � � �D�z�� � � �� �_�5�6� 6�� 9�C� �c�5�)�� �"8��c�� �CT� U�V�� � �3�x�$�-�-�-��K�� 7�8�8�
9� ��C��C��)�)�S�E�D�L�)�� �
�
�g� � �59�:�c�c�>�)�:�F�:� �S�t�F�T�J�F� ���a���A�3���F�
�
�
� � �V�$�� � � �� ;s �
D Nc �� � |rd|z }nd}| j rt t d� � S | j |� t di �� }| j
j
|� |j S )a�
Get some stats from the server. It will be available as a dict.
@param arg: An optional additional string which will be sent along
with the I{stats} command. The interpretation of this value by
the server is left undefined by the memcache protocol
specification.
@type arg: L{None} or L{bytes}
@return: a deferred that will fire with a L{dict} of the available
statistics.
@rtype: L{Deferred}
s stats rc r� )rP �rC r rd rG r r2 rL r )r" �argr; r� s r �statszMemCacheProtocol.stats� sd � � ��c�/�C��C� � � �� �_�5�6� 6� �
�
�c����"�-�� �
�
� � �V�$�� � � r c �� � | j rt t d� � S | j d� t d� }| j
j
|� |j S )z�
Get the version of the server.
@return: a deferred that will fire with the string value of the
version.
@rtype: L{Deferred}
r� s versionr� �r" r� s r �versionzMemCacheProtocol.version� sQ � � � � �� �_�5�6� 6� �
�
�j�!���$�� �
�
� � �V�$�� � � r c �4 � | j rt t d� � S t |t � s!t t dt
|� � d�� � S | j d|z � t d|�� }| j j |� |j S )a
Delete an existing C{key}.
@param key: the key to delete.
@type key: L{bytes}
@return: a deferred that will be called back with C{True} if the key
was successfully deleted, or C{False} if not.
@rtype: L{Deferred}
r� r� r� s delete s deleter� ) rC r rd r� r� r r� rG r r2 rL r )r" rf r� s r �deletezMemCacheProtocol.delete� s� � � � � �� �_�5�6� 6��#�u�%��� 4�T�#�Y�K�?P� Q�R��
�
�
�
�j�3�&�'����,�� �
�
� � �V�$�� � � r c �� � | j rt t d� � S | j d� t d� }| j
j
|� |j S )z�
Flush all cached values.
@return: a deferred that will be called back with C{True} when the
operation has succeeded.
@rtype: L{Deferred}
r� s flush_allr� r� s r �flushAllzMemCacheProtocol.flushAll� sQ � � � � �� �_�5�6� 6� �
�
�l�#�� �&�� �
�
� � �V�$�� � � r )�< )rt )r r )F)N)+r r r r r� rC r& r< rA rD rG rY r\ r_ rg ri rq ru rx r{ r� r� r� r� r� r� r� r� r� r� r� r� r� r� rL r� r� r� r� r� r� r� r� r r r r/ r/ c s� � ��2 �N��M� 8� �(�2�*�"�..� /��2/� �2 �5� "�'�'�.� .� /� "�:1�"1�$ �$G�.C�.C�.C�8 �>9�$:�$7�05�4 �< �2
� �,
r r/ )r/ � DEFAULT_PORTr
r r N)r � collectionsr �twisted.internet.deferr r r �twisted.protocols.basicr �twisted.protocols.policiesr �twisted.pythonr �twisted.python.compatr
r r� � Exceptionr
r r r r/ �__all__r r r �<module>r� sm ��
�0 � ?� ?� 0� 3� � =�� ��I� � �)� � �)� � &&� &&�Rq �|�\� q �h �r |