Do representations or analyses take precedence?

[iHiD]

If feedback is generated by both a representer and an analyzer, which set of feedback do we show? (I'm pretty sure it should only be one). Which is more "reliable"?

It's worth noting we should only hit this situation when an analyzer is added to an exercise after someone has commented on a representation.

[SleeplessByte]

Representer is human content. Probably that. But would be good to have a ping so the representation can be updated.

[iHiD]

But humans can be wrong? Whereas machines are always right? :)

[SleeplessByte]

Machines are as right as the humans whomstve programmed them ;)

[iHiD]

Yeah, so trust the programmers or the mentors. That's the question :)

[SleeplessByte]

Ah yes.

I'd show the handcrafted response by default, but I'd show the mentor the analyzer result. I'd ping whomever when the analyzer is updated so that the representation can be.

The mentor has "all" the context, the analyzer might not. Then again, I write pretty darn good analysers myself..mmmmm

[SaschaMann]

But humans can be wrong? Whereas machines are always right?

The analyzer would likely include the feedback given to the representation if it was created afterwards, wouldn't it? That would mean more humans were involved in the analyzer's feedback and it's likely higher quality.

[tehsphinx]

In my mind the analyzers are (at least for concept exercises) first and foremost an extension of the tests. They should point out things that tests can't like: "Prefer a switch here instead of if ... else if ... as there are 3 or more cases." (A preference in Go).

The implication of this approach is: A representation should be used if all tests pass AND the analyzer is happy.

On the other hand, if analyzers are used to put into rules what the representers/mentors would write:

• I can see how a mentors/representers output might be preferrable. If we think this way, then we should not try to generalize the rules of mentoring into analyzers but leave them to be extensions of tests only.
• I'd argue, that analyzer rules would be more consistent than all the different representations of many different mentors can ever be. Especially with time as preferences might change (in the language ecosystem itself or with the mentor community) it will be extremely hard to keep representations up to date.

With this approach I am not sure which is better but I slightly tend towards preferring the analyzers. For consistency.

[tehsphinx]

If we want to do both with the analyzers (use them as an extension of the tests AND use them to generalize rules) we might want to be able to differentiate the comments in the output of the analyzers.

[ErikSchierboom]

My personal preference is also to start with the analyzers. I agree with the analyzers being an extension of the tests (i.e. things that the tests can't verify), as well as having general rules that we don't want to encode in each test file (e.g. naming/white space rules).

Another reason why I would prefer analyzers to take precedence over representers is that they are usually relatively small, which means that students should be able to quickly fix them. Which makes for a nice "hey I can very easily improve my solution" feeling for students. Personally, I also love to make these little improvements that make the code just a tiny bit better. Representations can also do this, but I expect them to be slightly more verbose and possibly even a bit more contentious (the analyzers being more like rules).

IMHO the biggest of having the analyzers comment first is that all the common "mistakes" that are covered by analyzers will have be rooted out before passing them to the representer. This would make that the representer's problem space (the number of different solutions) is likely (much) smaller and there could be less representations needed to cover all the solutions.