pip_services3_data.persistence package

Submodules

• pip_services3_data.persistence.FilePersistence module
  • pip_services3_data.persistence.FilePersistence
• pip_services3_data.persistence.IdentifiableFilePersistence module
  • pip_services3_data.persistence.IdentifiableFilePersistence
• pip_services3_data.persistence.IdentifiableMemoryPersistence module
  • pip_services3_data.persistence.IdentifiableMemoryPersistence
• pip_services3_data.persistence.JsonFilePersister module
  • pip_services3_data.persistence.JsonFilePersister
• pip_services3_data.persistence.MemoryPersistence module
  • pip_services3_data.persistence.MemoryPersistence

Module contents

pip_services3_data.persistence.__init__

Pip-services-data.persistence initialisation.

Contains various persistence implementations (InMemory and File –persistences). These are”abstract” persistences, which only connect to data sources and do not implement the operations and methods for working the data. The classes that extend these persistences must implement this logic on their own.

Identifiable Persistences work with Identifiable objects, which have primary keys. A few standard operations are defined by default for these objects: reading arrays and data pages; searching for an object by its id; and creating, updating, and deleting records of objects.

copyright

Conceptual Vision Consulting LLC 2018-2019, see AUTHORS for more details.

license

MIT, see LICENSE for more details.

class pip_services3_data.persistence.FilePersistence(persister: Optional[pip_services3_data.persistence.JsonFilePersister.JsonFilePersister] = None)

Bases: pip_services3_data.persistence.MemoryPersistence.MemoryPersistence, pip_services3_commons.config.IConfigurable.IConfigurable

Abstract persistence component that stores data in flat files and caches them in memory.

This is the most basic persistence component that is only able to store data items of any type. Specific CRUD operations over the data items must be implemented in child classes by accessing self._items property and calling :func:save method.

### Configuration parameters ###

• path: path to the file where data is stored

### References ###

• *:logger:*:*:1.0 (optional) ILogger components to pass log messages

Example:

class MyJsonFilePersistence(FilePersistence):
    def __init__(self, path):
        super(MyJsonFilePersistence, self).__init__(JsonPersister(path))

    def get_by_name(self, correlationId, name):
        item = self.find(name)
        ...
        return item

configure(config: pip_services3_commons.config.ConfigParams.ConfigParams)

Configures component by passing configuration parameters.

Parameters

config – configuration parameters to be set.

class pip_services3_data.persistence.IdentifiableFilePersistence(persister: Optional[pip_services3_data.persistence.JsonFilePersister.JsonFilePersister] = None)

Bases: pip_services3_data.persistence.IdentifiableMemoryPersistence.IdentifiableMemoryPersistence, pip_services3_commons.data.IIdentifiable.IIdentifiable

Abstract persistence component that stores data in flat files and implements a number of CRUD operations over data items with unique ids. The data items must implement IIdentifiable interface.

In basic scenarios child classes shall only override get_page_by_filter(), get_list_by_filter() or delete_by_filter() operations with specific filter function. All other operations can be used out of the box. In complex scenarios child classes can implement additional operations by accessing cached items via self._items property and calling save() method on updates.

### Configuration parameters ###

• path: path to the file where data is stored

• options:

  • max_page_size: Maximum number of items returned in a single page (default: 100)

### References ###

• *:logger:*:*:1.0 (optional) ILogger components to pass log messages

Example:

class MyFilePersistence(IdentifiableFilePersistence):
    def __init__(self, path):
        super(MyFilePersistence, self).__init__(JsonPersister(path))

    def get_page_by_filter(self, correlationId, filter, paging):
        super().get_page_by_filter(correlationId, filter, paging, None)

    persistence = MyFilePersistence("./data/data.json")

    item = persistence.create("123", MyData("1", "ABC"))

    mydata = persistence.get_page_by_filter("123", FilterParams.from_tuples("name", "ABC"), None, None)
    print str(mydata.get_data())

    persistence.delete_by_id("123", "1")

configure(config: pip_services3_commons.config.ConfigParams.ConfigParams)

Configures component by passing configuration parameters.

Parameters

config – configuration parameters to be set.

class pip_services3_data.persistence.IdentifiableMemoryPersistence(loader: pip_services3_data.ILoader.ILoader = None, saver: pip_services3_data.ISaver.ISaver = None)

Bases: pip_services3_data.persistence.MemoryPersistence.MemoryPersistence, pip_services3_data.IWriter.IWriter, pip_services3_data.IGetter.IGetter, pip_services3_data.ISetter.ISetter, pip_services3_commons.data.IIdentifiable.IIdentifiable

Abstract persistence component that stores data in memory and implements a number of CRUD operations over data items with unique ids. The data items must implement IIdentifiable interface.

In basic scenarios child classes shall only override get_page_by_filter(), get_list_by_filter() or delete_by_filter() operations with specific filter function. All other operations can be used out of the box.

In complex scenarios child classes can implement additional operations by accessing cached items via this._items property and calling save() method on updates.

### Configuration parameters ###

• options:

  • max_page_size: Maximum number of items returned in a single page (default: 100)

### References ###

• *:logger:*:*:1.0 (optional) ILogger components to pass log messages

Example:

class MyMemoryPersistence(IdentifiableMemoryPersistence):

    def get_page_by_filter(self, correlationId, filter, paging):
        super().get_page_by_filter(correlationId, filter, paging, None)

    persistence = MyMemoryPersistence("./data/data.json")

    item = persistence.create("123", MyData("1", "ABC"))

    mydata = persistence.get_page_by_filter("123", FilterParams.from_tuples("name", "ABC"), None, None)
    print str(mydata.get_data())

    persistence.delete_by_id("123", "1")

create(correlation_id: Optional[str], item: T) → T

Creates a data item.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• item – an item to be created.

Returns

a created item

delete_by_id(correlation_id: Optional[str], id: Any) → T

Deleted a data item by it’s unique id.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• id – an id of the item to be deleted

Returns

a deleted item.

delete_by_ids(correlation_id: Optional[str], ids: List[Any])

Deletes multiple data items by their unique ids.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• ids – ids of data items to be deleted.

get_list_by_ids(correlation_id: Optional[str], ids: List[Any]) → List[T]

Gets a list of data items retrieved by given unique ids.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• ids – ids of data items to be retrieved

Returns

a data list of results by ids.

get_one_by_id(correlation_id: Optional[str], id: Any) → T

Gets a data item by its unique id.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• id – an id of data item to be retrieved.

Returns

data item by id.

id: Any

set(correlation_id: Optional[str], item: T) → T

Sets a data item. If the data item exists it updates it, otherwise it create a new data item.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• item – an item to be set.

Returns

an updated item

update(correlation_id: Optional[str], new_item: T) → T

Updates a data item.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• new_item – an item to be updated.

Returns

an updated item.

update_partially(correlation_id: Optional[str], id: Any, data: pip_services3_commons.data.AnyValueMap.AnyValueMap) → T

Updates only few selected fields in a data item.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• id – an id of data item to be updated.

• data – a map with fields to be updated.

Returns

an updated item.

class pip_services3_data.persistence.JsonFilePersister(path: str = None)

Bases: pip_services3_data.ILoader.ILoader, pip_services3_data.ISaver.ISaver, pip_services3_commons.config.IConfigurable.IConfigurable

Persistence component that loads and saves data from/to flat file.

It is used by FilePersistence, but can be useful on its own.

### Configuration parameters ###

• path: path to the file where data is stored

Example:

persister = JsonFilePersister("./data/data.json")

persister.save("123", ["A", "B", "C"])
...

persister.load("123", items)
print items

configure(config: pip_services3_commons.config.ConfigParams.ConfigParams)

Configures component by passing configuration parameters.

Parameters

config – configuration parameters to be set.

load(correlation_id: Optional[str]) → List[T]

Loads data items from external JSON file.

Parameters

correlation_id – (optional) transaction id to trace execution through call chain.

Returns

loaded items

property path

Gets the file path where data is stored.

Returns

the file path where data is stored.

save(correlation_id: Optional[str], items: List[T])

Saves given data items to external JSON file.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• items – list if data items to save

class pip_services3_data.persistence.MemoryPersistence(loader: pip_services3_data.ILoader.ILoader = None, saver: pip_services3_data.ISaver.ISaver = None)

Bases: pip_services3_commons.config.IConfigurable.IConfigurable, pip_services3_commons.refer.IReferenceable.IReferenceable, pip_services3_commons.run.IOpenable.IOpenable, pip_services3_commons.run.ICleanable.ICleanable

Abstract persistence component that stores data in memory.

This is the most basic persistence component that is only able to store data items of any type. Specific CRUD operations over the data items must be implemented in child classes by accessing self._items() property and calling save() method.

The component supports loading and saving items from another data source. That allows to use it as a base class for file and other types of persistence components that cache all data in memory.

### Configuration parameters ###

• options:

  • max_page_size: Maximum number of items returned in a single page (default: 100)

### References ###

• *:logger:*:*:1.0 (optional) ILogger components to pass log messages

Example:

class MyMemoryPersistence(MemoryPersistence):

    def get_by_name(self, correlationId, name):
        item = self.find(name)
        ...
        return item

persistence = MyMemoryPersistence()

persistence.set("123", MyData("ABC"))
print str(persistence.get_by_name("123", "ABC")))

clear(correlation_id: Optional[str])

Clears component state.

Parameters

correlation_id – (optional) transaction id to trace execution through call chain.

close(correlation_id: Optional[str])

Closes component and frees used resources.

Parameters

correlation_id – (optional) transaction id to trace execution through call chain.

configure(config: pip_services3_commons.config.ConfigParams.ConfigParams)

Configures component by passing configuration parameters.

Parameters

config – configuration parameters to be set.

create(correlation_id: Optional[str], item: T) → T

Creates a data item.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• item – an item to be created.

Returns

a created item

delete_by_filter(correlation_id: Optional[str], filter: Any)

Deletes data items that match to a given filter.

This method shall be called by a public delete_by_filter() method from child class that receives FilterParams and converts them into a filter function.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• filter – (optional) a filter function to filter items.

get_count_by_filter(correlation_id: Optional[str], filter: Any) → int

Gets a number of items retrieved by a given filter.

This method shall be called by a public get_count_by_filter method from child class that receives FilterParams and converts them into a filter function.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• filter – (optional) a filter function to filter items

Returns

a number of data items that satisfy the filter.

get_list_by_filter(correlation_id: Optional[str], filter: Any, sort: Any = None, select: Any = None) → List[T]

Gets a list of data items retrieved by a given filter and sorted according to sort parameters.

This method shall be called by a public get_list_by_filter() method from child class that receives FilterParams and converts them into a filter function.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• filter – (optional) a filter function to filter items

• sort – (optional) sorting parameters

• select – (optional) projection parameters (not used yet)

Returns

a data list of results by filter.

get_one_random(correlation_id: Optional[str]) → T

Gets a random item from items that match to a given filter.

This method shall be called by a public get_one_random() method from child class that receives FilterParams and converts them into a filter function.

Parameters

correlation_id – (optional) transaction id to trace execution through call chain.

Returns

a random item.

get_page_by_filter(correlation_id: Optional[str], filter: Any, paging: pip_services3_commons.data.PagingParams.PagingParams, sort: Any = None, select: Any = None) → pip_services3_commons.data.DataPage.DataPage

Gets a page of data items retrieved by a given filter and sorted according to sort parameters.

This method shall be called by a public get_page_by_filter() method from child class that receives FilterParams and converts them into a filter function.

Parameters

• correlation_id – (optional) transaction id to trace execution through call chain.

• filter – (optional) a filter function to filter items

• paging – (optional) paging parameters

• sort – (optional) sorting parameters

• select – (optional) projection parameters (not used yet)

Returns

a data page of result by filter.

is_open() → bool

Checks if the component is opened.

Returns

Anyrue if the component has been opened and false otherwise.

load(correlation_id: Optional[str])

TODO add description

open(correlation_id: Optional[str])

Opens the component.

Parameters

correlation_id – (optional) transaction id to trace execution through call chain.

save(correlation_id: Optional[str])

Saves items to external data source using configured saver component.

Parameters

correlation_id – (optional) transaction id to trace execution through call chain.

set_references(references: pip_services3_commons.refer.IReferences.IReferences)

Sets references to dependent components.

Parameters

references – references to locate the component dependencies.