Skip to content

Include record information in retries and fatals. #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lazamar merged 4 commits into from
Apr 23, 2026
Merged

Include record information in retries and fatals. #54

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6b2fc27
Include relevant fields in retry notification
lazamar Apr 23, 2026
de30c2d
Show raw record when decoding failure happens
lazamar Apr 23, 2026
aea0868
Truncate raw characters being shown
lazamar Apr 23, 2026
bc12bc0
Make it compile
lazamar Apr 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions src/Ambar/Emulator/Projector.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import qualified Data.ByteString.Lazy as LB
import Data.Maybe (listToMaybe, fromMaybe)
import Data.String (fromString)
import Data.Text (Text)
import qualified Data.Text.Encoding as Text
import qualified Data.Text as Text
import Control.Concurrent.Async (forConcurrently_)
import Control.Monad.Extra (whileM)
Expand Down Expand Up @@ -83,8 +84,9 @@ project logger_ Projection{..} =
Left err -> fatal logger (show err)
Right (bs, meta) -> do
record <- decode logger bs
retrying logger $ Transport.sendJSON p_transport (toMsg source record)
logInfo logger $ "sent. " <> relevantFields (s_source source) record
let logger' = annotate (relevantFields (s_source source) record) logger
retrying logger' $ Transport.sendJSON p_transport (toMsg source record)
logInfo logger' ("sent." :: Text)
Topic.commit consumer meta
return True

Expand All @@ -99,7 +101,12 @@ project logger_ Projection{..} =
}

decode logger bs = case Json.eitherDecode $ LB.fromStrict bs of
Left err -> fatal logger $ "decoding error: " <> err
Left err ->
let
raw_chars_to_show = 1000
raw = take raw_chars_to_show $ Text.unpack $ Text.decodeUtf8 bs
in
fatal logger $ "decoding error: " <> err <> "\nraw: " <> raw
Right v -> return v

-- | Fields to print when a record is sent.
Expand Down
Loading