Skip to content

Commit

Permalink
Further reorganize typeclass implicits for faster implicit search
Browse files Browse the repository at this point in the history
  - specific implicit for `Seq[T]`
  - make the implictits in the generated code by in a subclass
    of the low priority, expensive implicits.

After this change, we get to 187 discarded "expensive" implicits

    % scalac-hash v2.10.3 -Xlog-implicits  @args2.txt -Xprint:typer LME.scala 2>&1 | grep "not a valid" | wc -l

Down from 2200!

The benchmark compiles in 740ms, a 3.5x speedup from the original
2600ms.
  • Loading branch information
retronym committed Mar 24, 2014
1 parent a69576b commit a671b29
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 6 deletions.
4 changes: 2 additions & 2 deletions project/GenWireFormat.scala
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@ object GenWireFormat {
val place = dir / "scoobi" / "codegen" / "GeneratedWireFormats.scala"
IO.write(place,
"""package com.nicta.scoobi.codegen
import com.nicta.scoobi.core.WireFormat
import com.nicta.scoobi.core.{WireFormat, LowPriorityWireFormatImplicits}
import java.io._
object GeneratedWireFormats extends GeneratedWireFormats
trait GeneratedWireFormats {
trait GeneratedWireFormats extends LowPriorityWireFormatImplicits {
""" + gen_section_1(22) + "\n\n" + gen_section_2(22) + "\n}")

Seq(place)
Expand Down
6 changes: 4 additions & 2 deletions src/main/scala/com/nicta/scoobi/core/WireFormat.scala
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,10 @@ object WireFormat extends WireFormatImplicits {

// Deprioritize expensive implicits to improve compile times.
// See comments in `LowPriorityAvroSchemaImplicits`.
//
// Extended by `GeneratedWireFormats`
trait LowPriorityWireFormatImplicits {
self: WireFormatImplicits =>
import WireFormat._

/**
* Traversable structures
Expand Down Expand Up @@ -260,7 +262,7 @@ trait LowPriorityWireFormatImplicits {
}

/** Implicit definitions of WireFormat instances for common types. */
trait WireFormatImplicits extends LowPriorityWireFormatImplicits with codegen.GeneratedWireFormats {
trait WireFormatImplicits extends codegen.GeneratedWireFormats {

class ObjectWireFormat[T : Manifest](val x: T) extends WireFormat[T] {
override def toWire(obj: T, out: DataOutput) {}
Expand Down
8 changes: 6 additions & 2 deletions src/main/scala/com/nicta/scoobi/io/avro/AvroSchema.scala
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ trait AvroFixed[T] {
// even if one of these was found to be eligible, it would be deemed lower priority
// by the subclass rule of static overload resolution.
//
// These changes, together with corresponding changes in `WireFormat`, leads to a 2x
// speedup (2600ms ~> 1300ms) in compiling a benchmark:
// These changes, together with corresponding changes in `WireFormat`, leads to a 3.5x
// speedup (2600ms ~> 740ms) in compiling a benchmark:
// https://gist.github.com/retronym/fdef5a41c8e1e31124a4
//
// The goal is the reduce the number of lines like the following under `-Xlog-implicits`
Expand Down Expand Up @@ -193,6 +193,10 @@ object AvroSchema extends LowPriorityAvroSchemaImplicits {
def toAvro(x: String): String = x
}

// redundant, but better for performance to have this here rather than forcing
// this common implicit serach to go through TraversableSchema.
implicit def SeqSchema[T](implicit sch: AvroSchema[T]): AvroSchema[Seq[T]] = TraversableSchema[Seq, T]

/* AvroSchema type class instance for Arrays. */
implicit def ArraySchema[T](implicit mf: Manifest[T], sch: AvroSchema[T]) = new AvroSchema[Array[T]] {
type AvroType = GenericData.Array[sch.AvroType]
Expand Down

0 comments on commit a671b29

Please sign in to comment.