Skip to content
This repository was archived by the owner on Mar 20, 2024. It is now read-only.

Commit 689f293

Browse files
DanielSiepmannDanielSiepmann
committed
TASK: Finish deletion of index and documents feature
We replace the "flush" and "delete" by "delete" and "deletedocuments" logic. This makes it more obvious what will happen, without reading the docs. Also we kept the logic to always provide the index name, as we will need them in the future. Due to elasticsearch v6 changes no types are allowed in the same index in the future. Therefore we need to make it possible to use different indexes in the future, leading to the need to provide the document type all the time.
1 parent 28a8dd1 commit 689f293

File tree

15 files changed

+112
-94
lines changed

15 files changed

+112
-94
lines changed

‎Classes/Command/IndexCommandController.php‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -63,24 +63,24 @@ public function indexCommand(string $identifiers)
6363
*
6464
* @param string $identifier Comma separated list of identifiers.
6565
*/
66-
public function deleteCommand(string $identifiers)
66+
public function deleteDocumentsCommand(string $identifiers)
6767
{
6868
$this->executeForIdentifier($identifiers, function (IndexerInterface $indexer) {
69-
$indexer->deleteDocuments();
69+
$indexer->deleteAllDocuments();
7070
$this->outputLine('Documents in index ' . $indexer->getIdentifier() . ' were deleted.');
7171
});
7272
}
7373

7474
/**
75-
* Will flush the index for given identifiers from backend.
75+
* Will delete the index for given identifiers.
7676
*
7777
* @param string $identifier Comma separated list of identifiers.
7878
*/
79-
public function flushCommand(string $identifiers = 'pages')
79+
public function deleteCommand(string $identifiers = 'pages')
8080
{
8181
$this->executeForIdentifier($identifiers, function (IndexerInterface $indexer) {
8282
$indexer->delete();
83-
$this->outputLine('Indice ' . $indexer->getIdentifier() . ' was flushed.');
83+
$this->outputLine('Index ' . $indexer->getIdentifier() . ' was deleted.');
8484
});
8585
}
8686

‎Classes/Connection/ConnectionInterface.php‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -51,17 +51,17 @@ public function updateDocument(string $documentType, array $document);
5151
public function deleteDocument(string $documentType, string $identifier);
5252

5353
/**
54-
* Search by given request and return result.
54+
* Will all documents of certain kind / in certain index.
5555
*/
56-
public function search(SearchRequestInterface $searchRequest): SearchResultInterface;
56+
public function deleteAllDocuments(string $documentType);
5757

5858
/**
59-
* Will delete the index / db of defined document type.
59+
* Will delete the whole index / db.
6060
*/
61-
public function deleteIndexByDocumentType(string $documentType);
61+
public function deleteIndex(string $documentType);
6262

6363
/**
64-
* Will delete the whole index / db.
64+
* Search by given request and return result.
6565
*/
66-
public function deleteIndex();
66+
public function search(SearchRequestInterface $searchRequest): SearchResultInterface;
6767
}

‎Classes/Connection/Elasticsearch.php‎

Lines changed: 41 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,29 @@ function (Type $type, string $documentType) use ($identifier) {
142142
}
143143
}
144144

145+
public function deleteAllDocuments(string $documentType)
146+
{
147+
$this->deleteDocumentsByQuery($documentType, Query::create([
148+
'query' => [
149+
'term' => [
150+
'search_document_type' => $documentType,
151+
],
152+
],
153+
]));
154+
}
155+
156+
public function deleteIndex(string $documentType)
157+
{
158+
try {
159+
$this->indexFactory->getIndex($this->connection, $documentType)->delete();
160+
} catch (\InvalidArgumentException $e) {
161+
$this->logger->notice(
162+
'Index did not exist, therefore was not deleted.',
163+
[$documentType, $e]
164+
);
165+
}
166+
}
167+
145168
public function updateDocument(string $documentType, array $document)
146169
{
147170
$this->withType(
@@ -162,49 +185,6 @@ function (Type $type, string $documentType) use ($documents) {
162185
);
163186
}
164187

165-
public function deleteIndex()
166-
{
167-
$index = $this->connection->getClient()->getIndex($this->indexFactory->getIndexName());
168-
169-
if (!$index->exists()) {
170-
$this->logger->notice(
171-
'Index did not exist, therefore was not deleted.',
172-
[$this->indexFactory->getIndexName()]
173-
);
174-
return;
175-
}
176-
177-
$index->delete();
178-
}
179-
180-
public function deleteIndexByDocumentType(string $documentType)
181-
{
182-
$this->deleteIndexByQuery(Query::create([
183-
'query' => [
184-
'term' => [
185-
'search_document_type' => $documentType,
186-
],
187-
],
188-
]));
189-
}
190-
191-
private function deleteIndexByQuery(Query $query)
192-
{
193-
$index = $this->connection->getClient()->getIndex($this->indexFactory->getIndexName());
194-
if (!$index->exists()) {
195-
$this->logger->notice(
196-
'Index did not exist, therefore items can not be deleted by query.',
197-
[$this->indexFactory->getIndexName(), $query->getQuery()]
198-
);
199-
return;
200-
}
201-
$response = $index->deleteByQuery($query);
202-
if ($response->getData()['deleted'] > 0) {
203-
// Refresh index when delete query is invoked
204-
$index->refresh();
205-
}
206-
}
207-
208188
public function search(SearchRequestInterface $searchRequest): SearchResultInterface
209189
{
210190
$this->logger->debug('Search for', [$searchRequest->getSearchTerm()]);
@@ -232,4 +212,22 @@ private function withType(string $documentType, callable $callback)
232212
$callback($type, $documentType);
233213
$type->getIndex()->refresh();
234214
}
215+
216+
private function deleteDocumentsByQuery(string $documentType, Query $query)
217+
{
218+
try {
219+
$index = $this->indexFactory->getIndex($this->connection, $documentType);
220+
$response = $index->deleteByQuery($query);
221+
222+
if ($response->getData()['deleted'] > 0) {
223+
// Refresh index when delete query is invoked
224+
$index->refresh();
225+
}
226+
} catch (\InvalidArgumentException $e) {
227+
$this->logger->notice(
228+
'Index did not exist, therefore items can not be deleted by query.',
229+
[$documentType, $query->getQuery()]
230+
);
231+
}
232+
}
235233
}

‎Classes/Connection/Elasticsearch/IndexFactory.php‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -70,22 +70,35 @@ public function getIndexName(): string
7070
}
7171

7272
/**
73-
* Get an index based on TYPO3 table name.
73+
* @throws \InvalidArgumentException If index does not exist.
7474
*/
7575
public function getIndex(Connection $connection, string $documentType): \Elastica\Index
7676
{
7777
$index = $connection->getClient()->getIndex($this->getIndexName());
7878

7979
if ($index->exists() === false) {
80-
$config = $this->getConfigurationFor($documentType);
81-
$this->logger->debug(sprintf('Create index %s.', $documentType), [$documentType, $config]);
82-
$index->create($config);
83-
$this->logger->debug(sprintf('Created index %s.', $documentType), [$documentType]);
80+
throw new \InvalidArgumentException('The requested index does not exist.', 1546173102);
8481
}
8582

8683
return $index;
8784
}
8885

86+
public function createIndex(Connection $connection, string $documentType): \Elastica\Index
87+
{
88+
$index = $connection->getClient()->getIndex($this->getIndexName());
89+
90+
if ($index->exists() === true) {
91+
return $index;
92+
}
93+
94+
$config = $this->getConfigurationFor($documentType);
95+
$this->logger->debug(sprintf('Create index %s.', $documentType), [$documentType, $config]);
96+
$index->create($config);
97+
$this->logger->debug(sprintf('Created index %s.', $documentType), [$documentType]);
98+
99+
return $index;
100+
}
101+
89102
protected function getConfigurationFor(string $documentType): array
90103
{
91104
try {

‎Classes/Connection/Elasticsearch/TypeFactory.php‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,12 +49,9 @@ public function __construct(
4949
$this->connection = $connection;
5050
}
5151

52-
/**
53-
* Get an index bases on TYPO3 table name.
54-
*/
5552
public function getType(string $documentType): \Elastica\Type
5653
{
57-
$index = $this->indexFactory->getIndex($this->connection, $documentType);
54+
$index = $this->indexFactory->createIndex($this->connection, $documentType);
5855
return $index->getType('document');
5956
}
6057
}

‎Classes/Domain/Index/AbstractIndexer.php‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -106,14 +106,14 @@ public function indexDocument(string $identifier)
106106
public function delete()
107107
{
108108
$this->logger->info('Start deletion of index.');
109-
$this->connection->deleteIndex();
109+
$this->connection->deleteIndex($this->getDocumentName());
110110
$this->logger->info('Finish deletion.');
111111
}
112112

113-
public function deleteDocuments()
113+
public function deleteAllDocuments()
114114
{
115115
$this->logger->info('Start deletion of indexed documents.');
116-
$this->connection->deleteIndexByDocumentType($this->getDocumentName());
116+
$this->connection->deleteAllDocuments($this->getDocumentName());
117117
$this->logger->info('Finish deletion.');
118118
}
119119

‎Classes/Domain/Index/IndexerInterface.php‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,9 @@ public function indexDocument(string $identifier);
4242
public function delete();
4343

4444
/**
45-
* Delete the whole index.
45+
* Delete all documents from index.
4646
*/
47-
public function deleteDocuments();
47+
public function deleteAllDocuments();
4848

4949
/**
5050
* Receives the identifier of the indexer itself.

‎Documentation/source/changelog/0.1.0.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ v0.1.0
88
0.1.0/2018-changed-interfaces
99
0.1.0/2018-elasticsearch-upgrade
1010
0.1.0/2018-search-service-interface
11-
0.1.0/20181027-added-flush-command
11+
0.1.0/20181027-added-delete-all-documents-command
1212
0.1.0/20181027-allow-multiple-identifier-on-cli
1313
0.1.0/20181027-remove-cms7-support
1414
0.1.0/20181028-fluid-templating-list-items

‎Documentation/source/changelog/0.1.0/2018-changed-interfaces.rst‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Some interfaces and abstract classes have been adjusted:
55

66
``Codappix\SearchCore\Connection\ConnectionInterface``:
77

8-
* New method ``public function deleteIndexByDocumentType(string $documentType);``
8+
* New method ``public function deleteAllDocuments(string $documentType);``
99

1010
``Codappix\SearchCore\Domain\Index\IndexerInterface``:
1111

@@ -28,4 +28,12 @@ Also some exceptions have changed:
2828
throws an ``\InvalidArgumentException`` instead of ``\Exception``, if no
2929
``search_identifier`` was provided.
3030

31+
* ``Codappix\SearchCore\Connection\Elasticsearch\IndexFactory::getIndex()`` now
32+
throws an ``\InvalidArgumentException`` if the index does not exist. Leaving
33+
handling up to the caller.
3134

35+
Before the index was created if it didn't exist. To create an index, a new method
36+
``public function createIndex(Connection $connection, string $documentType): \Elastica\Index``
37+
was added. This method will only create the index if it didn't exist before.
38+
In the end, the index is returned always. Making this method a 1:1 replacement for
39+
older ``getIndex()``.

‎Documentation/source/changelog/0.1.0/20181027-added-delete-all-documents-command.rst‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Feature "Added delete documents command"
2+
========================================
3+
4+
A new command to delete all documents within an index was added. In contrast to the
5+
existing delete command, this deletes only documents but keeps the index.
6+
7+
E.g. if your backend is Elasticsearch or a relational database, the index or table is
8+
kept, including structure or mappings, while only the documents or rows are removed.
9+
10+
In contrast the existing delete command will still remove the index or table itself,
11+
depending on the used connection.

0 commit comments

Comments
 (0)