Imagine you're testing a service to creates, queries and deletes users. A fake version of that service might just be a wrapper on a HashMap keyed by ID. It might have several fields like some personal info, a hashed password, an email address, whether you're verified and so on.

Imagine one of your tests is if the user deletes their account. What pattern of calls should it make? You don't really care other than the record being deleted (or marked as deleted, depending on retention policy) after you're done.

In the mock world you might mock out calls like deleteUserByID and make suer it's called.

In the fake world, you simply check that the user record is deleted (or marked as such) after the test. You don't really care about what sequence of calls made that happen.

That may sound trivial but it gets less trivial the more complex your example is. Imagine instead you want to clear out all users who are marked for deletion. If you think about the SQL for that you might do a DELETE ... WHERE call so your API call might look like that. But if the logic is more complicated? Where if there's a change where EU and NA users have different retention periods or logging requirements so they're suddenly handled differently?

In a mokcing world you would have to change all your expected mocks. In fact, implementing this change might require fixing a ton of tests you don't care about at all and aren't really being broken by the change regardless.

In a fake world, you're testing what the data looks like after you're done, not the specific steps it took to get there.

Now those are pretty simple examples because there's not much to do the arguments used and no return values to speak of. Your code might branch differently based on those values, which then changes what calls to expects and with what values.

You're testing implementation details in a really time-consuming yet brittle way.

There are different kinds of mocks.

Check function XYZ is called, return abc when XYZ is called etc are the bad kind that people were bit badly by.

The good kind are a minimally correct fake implementation that doesn't really need any mocking library to build.

Tests should not be brittle and rigidly restate the order of function calls and expected responses. That's a whole lot of ceremony that doesn't really add confidence in the code because it does not catch many classes of errors, and requires pointless updates to match the implementation 1-1 everytime it is updated. It's effectively just writing the implementation twice, if you squint at it a bit.

The general term I prefer is test double. See https://martinfowler.com/bliki/TestDouble.html for how one might distinguish dummies, fakes, stubs, spies, and mocks.

Of course getting overly pedantic leads to its own issues, much like the distinctions between types of tests.

At my last Java job I used to commonly say things like "mocks are a smell", and avoided Mockito like GP, though it was occasionally useful. PowerMock was also sometimes used because it lets you get into the innards of anything without changing any code, but much more rarely. Ideally you don't need a test double at all.

Mocking is testing how an interface is used, rather than testing an implementation. That's why it requires some kind of library support. Otherwise you'd just on the hook for providing your own simple implementations of your dependencies.

Martin Fowler draws a useful distinction between mocks, fakes, and stubs¹. Fakes contain some amount of internal logic, e.g. a remote key-value store can be faked with a hashmap. Stubs are a bit dumber—they have no internal logic & just return pre-defined values. Mocks, though, are rigged to assert that certain calls were made with certain parameters. You write something like `myMock.Expect("sum").Args(1, 2).Returns(3)`, and then when you call `myMock.AssertExpectations()`, the test fails unless you called `myMock.sum(1, 2)` somewhere.

People often use the word "mock" to describe all of these things interchangeably², and mocking frameworks can be useful for writing stubs or fakes. However, I think it's important to distinguish between them, because tests that use mocks (as distinct from stubs and fakes) are tightly coupled to implementation, which makes them very fragile. Stubs are fine, and fakes are fine when stubs aren't enough, but mocks are just a bad idea.

[1]: https://martinfowler.com/articles/mocksArentStubs.html

[2]: The generic term Fowler prefers is "test double."

In part, you’re right, but there’s a practical difference between mocking and a good dummy version of a service. Take DynamoDB local as an example: you can insert items and they persist, delete items, delete tables, etc. Or in the Ruby on Rails world, one often would use SQLite as a local database for tests even if using a different DB in production.

Going further, there’s the whole test containers movement of having a real version of your dependency present for your tests. Of course, in a microservices world, bringing up the whole network of dependencies is extremely complicated and likely not warranted.

I'd go a bit farther — "mock" is basically the name for those dummy versions.

That said, there is a massive difference between writing mocks and using a mocking library like Mockito — just like there is a difference between using dependency injection and building your application around a DI framework.

I think the argument they're making is that once you have this, you already have an easy way to test things that doesn't require bringing in an entire framework.

The difference, IMO, between a mock and a proper "test" implementation is that traditionally a mock only exists to test interface boundaries, and the "implementation" is meant to be as much of a noop as possible. That's why the default behavior of almost any "automock" is to implement an interface by doing nothing and returning nothing (or perhaps default-initialized values) and provide tools for just tacking assertions onto it. If it was a proper implementation that just happened to be in-memory, it wouldn't really be a "mock", in my opinion.

For example, let's say you want to test that some handler is properly adding data to a cache. IMO the traditional mock approach that is supported by mocking libraries is to go take your RedisCache implementation and create a dummy that does nothing, then add assertions that say, the `set` method gets called with some set of arguments. You can add return values to the mock too, but I think this is mainly meant to be in service of just making the code run and not actually implementing anything.

Meanwhile, you could always make a minimal "test" implementation (I think these are sometimes called "fakes", traditionally, though I think this nomenclature is even more confusing) of your Cache interface that actually does behave like an in-memory cache, then your test could assert as to its contents. Doing this doesn't require a "mocking" library, and in this case, what you're making is not really a "mock" - it is, in fact, a full implementation of the interface, that you could use outside of tests (e.g. in a development server.) I think this can be a pretty good middle ground in some scenarios, especially since it plays along well with in-process tools like fake clocks/timers in languages like Go and JavaScript.

Despite the pitfalls, I mostly prefer to just use the actual implementations where possible, and for this I like testcontainers. Most webserver projects I write/work on naturally require a container runtime for development for other reasons, and testcontainers is glue that can use that existing container runtime setup (be it Docker or Podman) to pretty rapidly bootstrap test or dev service dependencies on-demand. With a little bit of manual effort, you can make it so that your normal test runner (e.g. `go test ./...`) can run tests normally, and automatically skip anything that requires a real service dependency in the event that there is no Docker socket available. (Though obviously, in a real setup, you'd also want a way to force the tests to be enabled, so that you can hopefully avoid an oopsie where CI isn't actually running your tests due to a regression.)

I'm not sure Kotlin is a hack, but it's growth is directly related to Java not being able to deliver on simple features that would help the day to day lives of developers. So I also can't blame anyone for moving on from the language. I get frustrated with it all the time, especially when Kotlin shows that some feature could work perfectly fine.

Here are some examples that have hit the graveyard: It's been 2 years since exception handling in switch was proposed [0], 3 years since null-restricted types were proposed [1], 4 years since string templates [2], 8 years since concise method bodies [3], and 11 years for JSON parsing [4].

[0] https://inside.java/2023/12/15/switch-case-effect/

[1] https://openjdk.org/jeps/8303099

[2] https://openjdk.org/jeps/430

[3] https://openjdk.org/jeps/8209434

[4] https://openjdk.org/jeps/198

Kotlin is a thoughtful language that is a joy to write.

Java was stagnant and ripe for being kicked off the top.

Scala was the hack that showed why Java was starting to suck but Scala suffered from no direction. Every single idea a PhD ever had was implemented in the language with no thought other than it seems cool to be able to do that too. Which is why all the Scala codebases fell apart, because you could write anything, anyway you want, and it was impossible to maintain without extremely strict guidelines on what parts of the language you were allowed to use. Also the build times were atrocious.

Kotlin designers evaluated language features across the ecosystem and chose what made sense to make Java better when it came out and now has easily surpassed Java. They are still thoughtful in what they choose to add to the language and it is now very powerful. However they smartly imitate Python by having clear guidelines on what choices you should be making when writing code so that other developers can easily dive in.

I used it mainly because the code I inherited was untestable as written. I made it testable via those methods. Then it got refactored.

Absolutely the worst. 1000 line test setups that shatter into pieces the instant you try to make the simplest change to a function. Makes refactoring an absolute nightmare.

Why? Every name you pick is likely to be weird in one language or another. Mockito does one thing well as a name, and that is hinting strongly at what it is (a mocking library).

Actually, no. "small booger" would be _moquito_ in spanish.

Most people want their test suite to pass. If they ugprade java and mockito prints out a message that they need to enabled '--some-flag' while running tests they're just going to add that flag to surefire in their pom. Seems like quite a small speedbump.

> Is there a point at which library maintainer feedback would meaningfully influence a by-default JVM change?

Of course, but keep in mind that all these changes were and are being done in response to feedback from other users, and we need to balance the requirements of mocking frameworks with those of people asking for better performance, better security, and better backward compatibility. When you have such a large ecosystem, users can have contradictory demands and sometimes it's impossible to satisfy everyone simultaneously. In those cases, we try to choose whatever we think will do the most good and the least harm over the entire ecosystem.

> JPMS is a good example: --add-opens still remains valuable today for important infra like Hadoop, Spark, and Netty. If other, even more core projects (e.g. Arrow) hadn't modernized, the exceptions would be even more prolific.

I think you have answered your own question. Make sure the libraries you rely on are well maintained, and if not - support them financially (actually, support them also if they are). BTW, I think that Netty is already in the process of abandoning its hacking of internals.

Anyone who has hacked internals agreed to a deal made in the notice we had in the internal files for many years prior to encapsulation [1], which was that the use of internals carries a commitment to added maintenance. Once they use the new supported mechanisms, that added burden is gone but they need to get there. I appreciate the fact that some open source projects are done by volunteers, and I think their users should compensate them, but they did enter into this deal voluntarily.

> If libraries so heavily depended upon like Mockito are unable to offer a viable alternative in response to JEP 451

But they have, and we advised them on how: https://github.com/mockito/mockito/issues/3037

The main "ergonomic" issue was lack of help from build tools like Gradle/Maven.

[1]: The notice was some version of:

    WARNING: The contents of this source file are not part of any supported API.
    Code that depends on them does so at its own risk: they are subject to change or removal without notice.

In September there was a supply-chain attack on NPM where the payload code injected hooks into the DOM API. Changing the behaviour of encapsulated components, like Java's standard library, is not possible now without the application explicitly allowing code to break the integrity of the encapsulated component.

Nothing makes it a massive headache, but I think that Maven/Gradle don't make it as easy as they could and should.

I haven't read the discussion but this seems like the obvious answer considering the flag only needs to be set during test.

Presumably this might miss some edge case (where something else also needs the flag?) though an explicit allow of the mockito agent in the jvm arg would have solved for that.

I agree. Tests relying on mocks rarely uncover or prevent issues. They also typically make it harder to make changes. Very bad idea that should have been left behind years ago.

How do you substitute for dependencies that you're not testing, or that you want to deliberately break?

Ok, can someone explain this to someone with double digit iq? Is it validating the adapter idea or something else?

Subclasing and overriding is not a good idea. There is no compilation failure if you forget to override a function which can lead to flakey tests at best and prod data impact at worst.

Google API libraries mark every class as "final" so it's not trivial to mock-extend it for tests. But third-party IO is exactly the thing you'd want to mock.

Probably because they zealously followed "Effective Java" book.

That’s the seductive power of mocking - you get a test up and running quickly. The benefit to the initial test writer is significant.

The cost is the pain - sometimes nightmarish - for other contributors to the code base since tests depending on mocking are far more brittle.

Someone changes code to check if the ResultSet is empty before further processing and a large number of your mock based tests break as the original test author will only have mocked enough of the class to support the current implementation.

Working on a 10+ year old code base, making a small simple safe change and then seeing a bunch of unit tests fail, my reaction is always “please let the failing tests not rely on mocks”.

An anonymous inner class is also ephemeral, declarative, inline, capable of extending as well as implementing, and readily readable. What it isn't is terse.

Mocking's killer feature is the ability to partially implement/extend by having some default that makes some sense in a testing situation and is easily instantiable without calling a super constructor.

Magicmock in python is the single best mocking library though, too many times have I really wanted mockito to also default to returning a mock instead of null.

What about paying your mortgage, food, utilities etc.? Or is that all supposed to come from savings or inheritance?