• 0 Posts
  • 49 Comments
Joined 1 year ago
cake
Cake day: July 14th, 2023

help-circle



  • I think part of the problem is that when you read about the horrors of the Holocaust as a kid, you can’t help but think of Nazi Germany as a cartoonishly, outlandishly evil place full of people who spend every waking second thinking about how much they hate impure bloodlines.

    You come away with an impression that it should be obvious when genocide is happening.

    Then you go home after school and you see something about genocide in the Middle East, and you ask your parents about it and they say “Well… it’s complicated.” And if it’s complicated – if it’s not cartoonishly, outlandishly evil – then it must not be genocide.




  • I tried to be accurate instead of specific.

    If I didn’t have to work anymore, I’d have more time to explore potential things to work on, so whatever project I’d pick right now would probably not be my main focus after 3 months of settling into my new life.

    From where I am right now, I think it would be something to do with language-level features for distributed computing (but not that web3 nonsense). There’s a lot of potential to weaken the monopoly power of cloud providers by working on something like that, which is why it’s an under-explored area.

    But I’d need more people to work with, and some specific use cases to go after. So I would expect the effort to change a lot by the time I actually found the right group of people to work with.





  • The venerable master Qc Na was walking with his student, Anton. Hoping to prompt the master into a discussion, Anton said “Master, I have heard that objects are a very good thing - is this true?” Qc Na looked pityingly at his student and replied, “Foolish pupil - objects are merely a poor man’s closures.”

    Chastised, Anton took his leave from his master and returned to his cell, intent on studying closures. He carefully read the entire “Lambda: The Ultimate…” series of papers and its cousins, and implemented a small Scheme interpreter with a closure-based object system. He learned much, and looked forward to informing his master of his progress.

    On his next walk with Qc Na, Anton attempted to impress his master by saying “Master, I have diligently studied the matter, and now understand that objects are truly a poor man’s closures.” Qc Na responded by hitting Anton with his stick, saying “When will you learn? Closures are a poor man’s object.” At that moment, Anton became enlightened.




  • kibiz0r@midwest.socialtoProgrammer Humor@programming.devLet's do micro service
    link
    fedilink
    English
    arrow-up
    4
    arrow-down
    1
    ·
    edit-2
    2 months ago

    I’m talking about user interactions, not deployments.

    In a monolith with a transactional data store, you can have a nice and clean atomic state transition from one complete, valid state to the next in a single request/response.

    With a distributed system, you’ll often have scenarios where the component which receives the initial request can’t guarantee the final state of the system by the time it needs to produce a response.

    If it did, it would spend most of its effort orchestrating other components. That would couple them together and be no more useful than a monolith, just with new and exciting failure modes. So really the best it can do is tell the client “Here’s a token you can use to check back on the state of this operation later”.

    And because data is often partitioned between different services, you can end up having partially-applied state changes. This leaves the data in an otherwise-invalid state, which must be accounted for – simply because of an implementation detail, not because it’s semantically meaningful to the client.

    In operations that have irreversible or non-idempotent external side-effects, this can be especially difficult to manage. You may want to allow the client to resume from immediately before or after the side-effect if there is a failure later on. Or you may want to schedule the side-effect, from the perspective of an earlier component in the chain, so that it happens even if a middle component fails (like the equivalent of a catch or finally block).

    If you try to cut corners by representing these things as special cases where the later components send data back to earlier ones, you end up introducing cycles in the data flow of your microservices. And then you’re in for a world of hurt. It’s better if you can represent it as a finite state machine, from the perspective of some coordinator component that’s not part of the data flow itself. But that’s a ton of work.

    It complicates every service that deals with it, and it gets really messy to just manage the data stores to track the state. And if you have queues and batching and throttling and everything else, along with granular permissions… Things can break. And they can break in really horrible ways, like infinitely sending the same data to an external service because the components keep tossing an event back to each other.

    There are general patterns – like sagas, distributed transactions, and event-sourcing – which can… kind of ease this problem. But they’re fundamentally limited by the CAP Theorem. And there isn’t a universally-accepted clean way to implement them, so you’re pretty much doing it from scratch each time.

    Don’t get me wrong. Sometimes “Here’s a token to check back later” and modeling interactions as a finite state machine rather than an all-or-nothing is the right call. Some interactions should work that way. But you should build them that way on purpose, not to work around the downsides of a cool buzzword you decided to play around with.


  • kibiz0r@midwest.socialtoProgrammer Humor@programming.devLet's do micro service
    link
    fedilink
    English
    arrow-up
    11
    arrow-down
    1
    ·
    2 months ago

    Microservices can be useful, but yeah working in a codebase where every little function ends up having to make a CAP Theorem trade-off is exhausting, and creates sooo many weird UX situations.

    I’m sure tooling will mature over time to ease the pain of representing in-flight, rolling-back, undone, etc. states across an entire system, but right now it feels like doing reactive programming without observables.

    And also just… not everything needs to scale like whoa. And they can scale in different ways: queue up-front, data replication afterwards, syncing ledgers of CRDTs… Scaling in-flight operations is often the worst option. But it feels familiar, so it’s often the default choice.




  • I need help finding a source, cuz there are so many fluff articles about medical AI out there…

    I recall that one of the medical AIs that the cancer VC gremlins have been hyping turned out to have horribly biased training data. They had scans of cancer vs. not-cancer, but they were from completely different models of scanners. So instead of being calibrated to identify cancer, it became calibrated to identify what model of scanner took the scan.