NeptuneData / Client / get_propertygraph_stream

get_propertygraph_stream#

NeptuneData.Client.get_propertygraph_stream(**kwargs)#

Gets a stream for a property graph.

With the Neptune Streams feature, you can generate a complete sequence of change-log entries that record every change made to your graph data as it happens. GetPropertygraphStream lets you collect these change-log entries for a property graph.

The Neptune streams feature needs to be enabled on your Neptune DBcluster. To enable streams, set the neptune_streams DB cluster parameter to 1.

See Capturing graph changes in real time using Neptune streams.

When invoking this operation in a Neptune cluster that has IAM authentication enabled, the IAM user or role making the request must have a policy attached that allows the neptune-db:GetStreamRecords IAM action in that cluster.

When invoking this operation in a Neptune cluster that has IAM authentication enabled, the IAM user or role making the request must have a policy attached that enables one of the following IAM actions, depending on the query:

Note that you can restrict property-graph queries using the following IAM context keys:

See Condition keys available in Neptune IAM data-access policy statements).

See also: AWS API Documentation

Request Syntax

response = client.get_propertygraph_stream(
    limit=123,
    iteratorType='AT_SEQUENCE_NUMBER'|'AFTER_SEQUENCE_NUMBER'|'TRIM_HORIZON'|'LATEST',
    commitNum=123,
    opNum=123,
    encoding='gzip'
)
Parameters:

  • limit (integer) –

    Specifies the maximum number of records to return. There is also a size limit of 10 MB on the response that can’t be modified and that takes precedence over the number of records specified in the limit parameter. The response does include a threshold-breaching record if the 10 MB limit was reached.

    The range for limit is 1 to 100,000, with a default of 10.

  • iteratorType (string) –

    Can be one of:

    • AT_SEQUENCE_NUMBER   – Indicates that reading should start from the event sequence number specified jointly by the commitNum and opNum parameters.

    • AFTER_SEQUENCE_NUMBER   – Indicates that reading should start right after the event sequence number specified jointly by the commitNum and opNum parameters.

    • TRIM_HORIZON   – Indicates that reading should start at the last untrimmed record in the system, which is the oldest unexpired (not yet deleted) record in the change-log stream.

    • LATEST   – Indicates that reading should start at the most recent record in the system, which is the latest unexpired (not yet deleted) record in the change-log stream.

  • commitNum (integer) – The commit number of the starting record to read from the change-log stream. This parameter is required when iteratorType is AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER, and ignored when iteratorType is TRIM_HORIZON or LATEST.

  • opNum (integer) – The operation sequence number within the specified commit to start reading from in the change-log stream data. The default is 1.

  • encoding (string) – If set to TRUE, Neptune compresses the response using gzip encoding.

Return type:

dict

Returns:

Response Syntax

{
    'lastEventId': {
        'string': 'string'
    },
    'lastTrxTimestampInMillis': 123,
    'format': 'string',
    'records': [
        {
            'commitTimestampInMillis': 123,
            'eventId': {
                'string': 'string'
            },
            'data': {
                'id': 'string',
                'type': 'string',
                'key': 'string',
                'value': {...}|[...]|123|123.4|'string'|True|None,
                'from': 'string',
                'to': 'string'
            },
            'op': 'string',
            'isLastOp': True|False
        },
    ],
    'totalRecords': 123
}

Response Structure

  • (dict) –

    • lastEventId (dict) –

      Sequence identifier of the last change in the stream response.

      An event ID is composed of two fields: a commitNum, which identifies a transaction that changed the graph, and an opNum, which identifies a specific operation within that transaction:

      • (string) –

        • (string) –

    • lastTrxTimestampInMillis (integer) –

      The time at which the commit for the transaction was requested, in milliseconds from the Unix epoch.

    • format (string) –

      Serialization format for the change records being returned. Currently, the only supported value is PG_JSON.

    • records (list) –

      An array of serialized change-log stream records included in the response.

      • (dict) –

        Structure of a property graph record.

        • commitTimestampInMillis (integer) –

          The time at which the commit for the transaction was requested, in milliseconds from the Unix epoch.

        • eventId (dict) –

          The sequence identifier of the stream change record.

          • (string) –

            • (string) –

        • data (dict) –

          The serialized Gremlin or openCypher change record.

          • id (string) –

            The ID of the Gremlin or openCypher element.

          • type (string) –

            The type of this Gremlin or openCypher element. Must be one of:

            • v1   - Vertex label for Gremlin, or node label for openCypher.

            • vp   - Vertex properties for Gremlin, or node properties for openCypher.

            • e   - Edge and edge label for Gremlin, or relationship and relationship type for openCypher.

            • ep   - Edge properties for Gremlin, or relationship properties for openCypher.

          • key (string) –

            The property name. For element labels, this is label.

          • value (document) –

            This is a JSON object that contains a value field for the value itself, and a datatype field for the JSON data type of that value:

          • from (string) –

            If this is an edge (type = e), the ID of the corresponding from vertex or source node.

          • to (string) –

            If this is an edge (type = e), the ID of the corresponding to vertex or target node.

        • op (string) –

          The operation that created the change.

        • isLastOp (boolean) –

          Only present if this operation is the last one in its transaction. If present, it is set to true. It is useful for ensuring that an entire transaction is consumed.

    • totalRecords (integer) –

      The total number of records in the response.

Exceptions