Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#245 Add the ability to query REST endpoints from Reader module #297

Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
b97b603
#244: Create the Info module
benedeki Aug 19, 2024
e623974
* fixed License headers
benedeki Aug 19, 2024
5e4eadb
* renamed to _Reader_
benedeki Aug 24, 2024
2e1e2ea
* README.md update
benedeki Aug 25, 2024
738c904
* fix
benedeki Aug 25, 2024
df8c9bd
* JaCoCO action update
benedeki Aug 26, 2024
5affd82
* added dummy code for testing coverage
benedeki Aug 26, 2024
0f1e121
* erroneous class renamed
benedeki Aug 26, 2024
d773a93
* Deleted wrong files
benedeki Aug 26, 2024
0776f9c
#245 Add the ability to query REST endpoints from Reader module
benedeki Sep 10, 2024
38fde1c
* Work still in progress
benedeki Sep 23, 2024
1ac2233
* the first working commit
benedeki Nov 1, 2024
e6dcb52
* Removed temporary notes
benedeki Nov 1, 2024
6968b02
* introduced `MonadError` into the `GenericServerConnection`
benedeki Nov 1, 2024
b9bacef
* Fixed UTs
benedeki Nov 1, 2024
bbb1e7f
* trying to get rid of Java 11 dependency
benedeki Nov 4, 2024
33e6628
* Downgraded sttpClient
benedeki Nov 4, 2024
f7ced56
* further downgrade
benedeki Nov 5, 2024
ca2116b
* Removed exceptions
benedeki Nov 6, 2024
e5e6f63
* commented out parts of README.md which are not yet part of the code
benedeki Nov 6, 2024
fe07272
- major rework
benedeki Nov 17, 2024
7656f6f
* doc fix
benedeki Nov 17, 2024
eb9a678
Merge branch 'master' into feature/245-add-the-ability-to-query-rest-…
benedeki Nov 17, 2024
7641c07
* disabled failing test
benedeki Nov 17, 2024
bc82a5b
* adjustments
benedeki Nov 18, 2024
0e7675e
- further cleaning
benedeki Nov 18, 2024
432716a
* tests progress
benedeki Nov 21, 2024
11b0a16
* several UTs added
benedeki Nov 22, 2024
2c3f145
Merge branch 'master' into feature/245-add-the-ability-to-query-rest-…
benedeki Nov 22, 2024
e07dffb
* last improvements before PR ready
benedeki Nov 24, 2024
3955a50
* description to class `ServerConfig`
benedeki Nov 25, 2024
b287a66
* removed empty line
benedeki Nov 25, 2024
d04d23b
* addressed PR comments
benedeki Nov 27, 2024
c344249
* just better implementation
benedeki Nov 30, 2024
a4759d1
Apply suggestions from code review
benedeki Dec 4, 2024
5ca4bd5
* small fixes
benedeki Dec 4, 2024
d5054b6
Merge branch 'master' into feature/245-add-the-ability-to-query-rest-…
benedeki Dec 4, 2024
f8c1d86
Apply suggestions from code review
benedeki Dec 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,4 @@ class FlowReader[F[_]](val mainFlowPartitioning: AtumPartitions)
(implicit serverConfig: ServerConfig, backend: SttpBackend[F, Any], ev: MonadError[F])
extends Reader[F] with PartitioningIdProvider[F]{

override def partitioning: AtumPartitions = mainFlowPartitioning

}
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,7 @@ import za.co.absa.atum.model.utils.JsonSyntaxExtensions.JsonSerializationSyntax
import za.co.absa.atum.reader.basic.RequestResult.RequestResult

trait PartitioningIdProvider[F[_]] {self: Reader[F] =>
def partitioning: AtumPartitions

def partitioningId()(implicit monad: MonadError[F]): F[RequestResult[Long]] = {
def partitioningId(partitioning: AtumPartitions)(implicit monad: MonadError[F]): F[RequestResult[Long]] = {
val encodedPartitioning = partitioning.toPartitioningDTO.asBase64EncodedJsonString
val queryResult = getQuery[SingleSuccessResponse[PartitioningWithIdDTO]]("/api/v2/partitionings", Map("partitioning" -> encodedPartitioning))
queryResult.map{result =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,20 +58,22 @@ class PartitioningIdProviderUnitTests extends AnyFunSuiteLike {
(implicit serverConfig: ServerConfig)
extends Reader[Identity] with PartitioningIdProvider[Identity]{

override def partitioningId()(implicit monad: MonadError[Identity]): Identity[RequestResult[Long]] = super.partitioningId()
override def partitioningId(partitioning: AtumPartitions)
(implicit monad: MonadError[Identity]): Identity[RequestResult[Long]] =
super.partitioningId(partitioning)
}


test("Gets the partitioning id") {
val reader = ReaderWithPartitioningIdForTest(atumPartitionsToReply)
val response = reader.partitioningId()
val response = reader.partitioningId(atumPartitionsToReply)
val result: Long = response.getOrElse(throw new Exception("Failed to get partitioning id"))
assert(result == 1)
}

test("Not found on the partitioning id") {
val reader = ReaderWithPartitioningIdForTest(atumPartitionsToNotFound)
val result = reader.partitioningId()
val result = reader.partitioningId(atumPartitionsToNotFound)
result match {
case Right(_) => fail("Expected a failure, but OK response received")
case Left(_: DeserializationException[CirceError]) => fail("Expected a not found response, but deserialization error received")
Expand All @@ -82,9 +84,10 @@ class PartitioningIdProviderUnitTests extends AnyFunSuiteLike {
}
}

test("Failure to decode response body") {
test("Failure to decode res " +
salamonpavel marked this conversation as resolved.
Show resolved Hide resolved
"]ponse body") {
val reader = ReaderWithPartitioningIdForTest(atumPartitionsToFailedDecode)
val result = reader.partitioningId()
val result = reader.partitioningId(atumPartitionsToFailedDecode)
assert(result.isLeft)
result.swap.map(e => assert(e.isInstanceOf[DeserializationException[CirceError]]))
}
Expand Down
Loading