Fuzzy matching of words with phrases with ruby -


i want match bunch of data short number of services

my data this

{"title" : "blorb", "category" : "zurb" "description" : "massage manipulation of superficial , deeper layers of muscle , connective tissue using various techniques, enhance function, aid in healing process, decrease muscle reflex activity..." }

and have match

["swedish massage", "haircut"]

clearly "swedish massage" winner, running benchmark shows "haircut" is:

require 'amatch'  arr = [:levenshtein_similar, :hamming_similar, :pair_distance_similar, :longest_subsequence_similar, :longest_substring_similar, :jaro_similar, :jarowinkler_similar]  arr.each |method|   ["swedish massage", "haircut"].each |sh|     pp ">>> #{sh} matched #{method.to_s}"     pp sh.send(method, description)   end end , nil

result:

">>> swedish massage matched jaro_similar" # 0.5246896118183247 ">>> haircut matched jaro_similar" # 0.5353606789250354 ">>> swedish massage matched jarowinkler_similar" # 0.5246896118183247 ">>> haircut matched jarowinkler_similar" # 0.5353606789250354

the rest of indices below 0.1

what better approach solving problem?

search constant battle between precision , recall. 1 thing try splitting input words - result in stronger match on massage consequence of broadening out result set. find sentences returned words close swedish. try control broadening averaging results multiple words, using stop lists avoid common words and, boosts finding tokens close each other etc, never see perfect results. if you're interested in fine tuning recommend elasticsearch - relatively easy learn , powerful.


-

Comments

Post a Comment

Popular posts from this blog

php - regexp cyrillic filename not matches -

i take list of files php glob(."*.pdf") . files have cyrillic names. example: ООО «Рога и копыта» then load company names database , try find them in file names: preg_match("/$firm_name/ui", $file_name, $match); i have problem: firms if $firm_name visually similar $file_name , preg_match says there no match. mb_detect_encoding both variables return utf-8. there problem in file name, or maybe there encoding or something, can not understand, ask help. upd: $file_name='ООО «Рога и копыта»'; preg_match("/... .Рога и копыта./ui", $file_name, $match); as result, ОО «Рога и копыта» , 1 О missing
Read more

c# - OpenXML hanging while writing elements -

Image
i have program pulls data database, caches file , exports data multiple formats (excel, excel 2003, csv). i'm using openxml sdk 2.0 excel work. these export processes run in parallel (using parallel.foreach ), , amount of data can pretty large - e.g. csvs 800mb. during these larger exports, i've noticed writing of xml documents hang. instance, if have 8 exporting in parallel, @ point "pause". hang around same point: //this.writer openxmlwriter created worksheetpart. this.writer.writeelement(new cell() { cellvalue = new cellvalue(value), datatype = cellvalues.string }); when happens, pause debugger (vs2013 in case) , notice threads blocking around same portion of code - bit deeper in openxml sdk - stem call openxmlwriter.writeelement . i dug through source using justdecompile didn't find answers. appears there intermediary stream in use writing isolated storage , is, reason, blocking. underlying stream each of these filestream . here screenshot ...
Read more

sql - Select Query has unexpected multiple records (MS Access) -

i may tired, can't seem figure out happening. have 2 queries. query 1 has 10 fields. query 2 has 5 fields. fields 1 through 3 same in both queries. trying write select query , simple add field 5 query2 query1 result should query1 additional field (query2.field5). i join first 3 fields both queries , choose select records in query1 , in query2 match. don't know sql, use query designer. expect same number of records have query 1 instead have 3 times or more. can tell me doing wrong. tried reversing joins still seem same records. thanks in advance. here query: the fields in common divisionname , supplierid = vendorid , commodityname . select [part revenue exposed query p1].divisionname, [part revenue exposed query p1].commodityname, [part revenue exposed query p1].suppliername, [part revenue exposed query p1].partnumber, [part revenue exposed query p1].partdescription, [part revenue exposed query p1].bucode, ...
Read more