Building project for the first time in IntelliJ 2021 experiences

[nezda]

It took me a while to get the project to build in IntelliJ 2021.1 but eventually I got it going... mostly. Here are some of my experiences:

• use of https://sdkman.io/ made installing java 11, scala 2.12, and sbt 1.2.7 really easy - I recommend that to everyone for local development (less cool for CI and servers)
• build seems to fail with sbt 1.4.9 because the following uses http instead of https:

  odinson/extra/build.sbt

  Line 3 in 89277aa

  resolvers += "Artifactory" at "http://artifactory.cs.arizona.edu:8081/artifactory/sbt-release"

  : downgrading to sbt 1.2.7 solved
• BuildInfo didn't work until I switched on sbt builds https://www.jetbrains.com/help/idea/compile-and-build-scala-projects.html#scala_delegate
• IntelliJ says "Info: sbt compilation for play framework 2.x disabled by default" - is that used in this project?
• Even now IntelliJ is a little confused on the file I happen to be on:

  odinson/core/src/main/scala/ai/lum/odinson/OdinsonIndexWriter.scala

  Lines 252 to 278 in 1c5394e

  	def fromConfig(config: Config): OdinsonIndexWriter = {
  	val indexDir = config[String]("odinson.indexDir")
  	val documentIdField = config[String]("odinson.index.documentIdField")
  	val sentenceIdField = config[String]("odinson.index.sentenceIdField")
  	val sentenceLengthField = config[String]("odinson.index.sentenceLengthField")
  	val normalizedTokenField = config[String]("odinson.index.normalizedTokenField")
  	val addToNormalizedField = config[List[String]]("odinson.index.addToNormalizedField")
  	val incomingTokenField = config[String]("odinson.index.incomingTokenField")
  	val outgoingTokenField = config[String]("odinson.index.outgoingTokenField")
  	val maxNumberOfTokensPerSentence = config[Int]("odinson.index.maxNumberOfTokensPerSentence")
  	val invalidCharacterReplacement = config[String]("odinson.index.invalidCharacterReplacement")
  	val storedFields = config[List[String]]("odinson.index.storedFields")
  	val displayField = config[String]("odinson.displayField")
  	val (directory, vocabulary) = indexDir match {
  	case ":memory:" =>
  	// memory index is supported in the configuration file
  	val dir = new RAMDirectory
  	val vocab = Vocabulary.empty
  	(dir, vocab)
  	case path =>
  	val dir = FSDirectory.open(new File(path).toPath)
  	val vocab = Vocabulary.fromDirectory(dir)
  	(dir, vocab)
  	}
  	// Always store the display field, also store these additional fields
  	if (!storedFields.contains(displayField)) {
  	throw new OdinsonException("`odinson.index.storedFields` must contain `odinson.displayField`")

Ambiguous implicits..., No implicits found for evidence..., Cannot resolve overloaded constructor 'File', Cannot resolve symbol !, Cannot resolve symbol contains. What's misconfigured in my IntelliJ?

Maybe something in

Next file I went to had more type complaints:

odinson/extra/src/main/scala/ai/lum/odinson/extra/IndexDocuments.scala

Lines 42 to 70 in ac6ca7b

	val docsDir = config[File]("odinson.docsDir")
	val synchronizeOrderWithDocumentId =
	config[Boolean]("odinson.index.synchronizeOrderWithDocumentId")
	//
	val writer = OdinsonIndexWriter.fromConfig(config)
	val wildcards = Seq("*.json", "*.json.gz")
	logger.info(s"Gathering documents from $docsDir")
	// make this a function
	val documentFiles =
	if (synchronizeOrderWithDocumentId) {
	// files ordered by the id of the document
	docsDir
	.listFilesByWildcards(wildcards, recursive = true)
	.map(f => (Document.fromJson(f).id.toInt, f))
	.toSeq
	.sortBy(_._1)
	.map(_._2)
	} else {
	docsDir
	.listFilesByWildcards(wildcards, recursive = true)
	.par
	}
	// ^ this part should be a function
	logger.info("Indexing documents")
	indexDocuments(writer, documentFiles)

It would be nice to get rid of these red squiggles.

Some of this might be useful in the docs. Looks like a fascinating project - going to dig in a bit now!

Kind regards,

• Luke

[BeckySharp]

Thanks! I also use intelliJ and don't have the same squiggles, BUT intelliJ is confused when I use the ai.lum.common.ConfigUtils._ implicits on the Config object
(to do stuff like: val x = config[String]("myCoolPath"))

Out of curiosity, if on line 42 you add the explicit type declaration (val docsDir: File = ...) do the remaining resolve?
That's something I do in my own projects when using that library to reduce squiggles.

Here's my scala compiler screed fwiw

@kwalcock any additional thoughts, if you have a chance?

[kwalcock]

Be sure to use 2021.1.1. There were lots of problems with 2021.1.0.

If you want to upgrade to something other than sbt.version=1.2.6, you'll have to account for it. Higher versions require something like

("Artifactory" at "http://artifactory.cs.arizona.edu:8081/artifactory/sbt-release" ).withAllowInsecureProtocol(true)

IntelliJ is just not good with implicits, especially those config[] ones. I get the squiggles as well in all versions of IntelliJ.

For the code generation associated with BuildInfo, it seems to help to run sbt once in advance. IntelliJ seems to get the idea afterwards. There are some checks that can be made in the Project Structure settings in the Modules. It seems like the source folders that get found aren't always right and that they might be influenced by previous builds.

[nezda]

"get the idea" - heh. I'm on IntelliJ IDEA 2021.1.1 .

seems to help to run sbt once in advance
that was my experience as well. Maybe some build docs could say that?

Bummer about the red squiggles...

My line 45 is val docsDir: File = config[File]("odinson.docsDir"): no squiggles. Line 46 is first squiggles in that file val processorType = config[String]("odinson.extra.processorType"): looks like type is explicit?

over the pipe

[nezda]

ah, line 46 val processorType: String = config[String]("odinson.extra.processorType") got rid of that squiggle.

line 24 val processor: String = if (args.length == 2) args(1) else config[String]("odinson.extra.processorType") got the other in AnnotateText

[nezda]

Shall I make a PR?

[BeckySharp]

I think some argue that it’s IntelliJ’s problem, esp those who code in vim :) if you don’t mind, let me bring it up at the next odinson technical meeting and Ill see if there’s “style” lum wants to go with when the issue is ~ with the ide

[BeckySharp]

fwiw i’m with you and squiggles make me sad

[nezda]

I coded in vim for many years - I kinda get it

[kwalcock]

If the squiggles irritate you too much, you can change code like

    val baseGrounder = BaseGrounder.fromConfig(config[Config]("eventBase"), stopwordManaging)

to

    val baseGrounder = BaseGrounder.fromConfig(config.apply[Config]("eventBase"), stopwordManaging)

by adding the apply. Otherwise, it seems like the proper place to report the error is at JetBrains. IntelliJ has a help menu item that takes you right there.

[nezda]

So you're good with adding implied apply but not cast-like type specification right @kwalcock ? (that mostly makes sense to me I just want to make sure I understand your perspective) Obviously you're right - its an Intellij bug; https://youtrack.jetbrains.com/issue/SCL-17529 already open seems to cover it so I just voted for it. I wonder what proportion of your team and users is impacted by it? I think they should vote for that issue too (or the right one if that one is wrong): I feel like on-the-fly error flagging is one of the things that makes coding with a statically typed language in an IDE so productive, so false positives are a drain in that sense.

[kwalcock]

Yes, but in addition, the type specification didn't seem sufficient in some cases. That may happen with apply as well. I think it's better not to scratch this one.

[BeckySharp]

Many in the team want to fix this. I spoke with Marco and I think we’re hoping to fix it at the source (i.e., ai.lum.common), else any of these solutions are ok.

…

On Thursday, May 20, 2021, Keith Alcock ***@***.***> wrote: *External Email* Yes, but in addition, the type specification didn't seem sufficient in some cases. That may happen with apply as well. I think it's better not to scratch this one. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#289 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJCPCNCDQSBFZ2LCUQUMQ3TOTDF7ANCNFSM444I3GSQ> .

[nezda]

I tried briefly to refresh my knowledge of implicits but didn't immediately see solution and aborted my effort.

[BeckySharp]

So @nezda -- we found a way to rework the source code, but a huge problem was that the type checking moved from compile time to run time, which we feel is too big of a loss.
So, if you feel motivated (you mentioned above opening a PR, hope I'm not over-presuming) to convert the implicit calls to the versions with apply -- e.g., config.apply[String](...) that would be great and (much) welcomed.

thoughts?

[kwalcock]

Apply and implicit are in store for some very significant changes in Scala 3. I wonder whether that would be a better target. It may be that whatever we change now will have to change again soon if we're not careful.

[nezda]

I understand. Shouldn't be too bad to regex this back out: #291