Skip to content

Conversation

jkinkead
Copy link

Fixes #28 by splitting large models into multiple buffers in memory.

Note on formatting: I used tabs throughout, as that seemed to be more common in the files. SearcherImpl was mixed-use. I also tried to keep for-loop (for( vs for () and naming similar to the local code. This was mixed-use as well.

@jkinkead jkinkead mentioned this pull request Aug 24, 2015
@dirkgr
Copy link
Contributor

dirkgr commented Sep 23, 2015

Ping?

@krishnad
Copy link

This fork breaks in the toBinFile method.
for(int i = 0; i < vocab.size(); ++i) {
out.write(String.format("%s ", vocab.get(i)).getBytes(cs));

        DoubleBuffer vectorBuffer = vectors[i / vectorsPerBuffer];

vectors ( with the plural) is a DoubleBuffer[] and is unlikely to be as large as the vocab size. In most of my test cases, vectors is a DoubleBuffer array of length 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants