MIR chapter10 User interfaces and visualisation
1. Human-Computer Interaction:
Information access(design principles): provide informative feedback, permit easy reversal of actions, support an internal locus of control, reduce working memory load, and provide alternative interfaces for novice and expert users.
Information visualisation: to provide visual depictions of very large information spaces, using icons and colour highlighting, brushing and linking, panning and zooming, focus-plus-context, magic lenses and animation to retain context and help make occluded information visible.
2. Evaluate interactive system:
Precision and recall measures have been widely used for comparing the ranking results of non-interactive systems, but are less appropriate for assessing interactive systems.
3. Information access process: an interaction cycle consisting of query specification, receipt and examination of retrieval results and then either stopping or reformulating the query and repeating the process until a perfect result set is found. How to reformulate the query?
4. Starting points: lists, overviews, and automated source selection.
5. Query specification: command language, form fill in, menu selection, direct manipulation, and nature language.-boolean query specification.
Friday, February 28, 2014
Week8——Muddiest Points(unit 7)
Unit7: relevance feedback& query expansion
1. How to decide the value of trade-off between recall and precision?
2. Evaluate ranked retrieval:
The average values may have different performances, so how we judge the system is stable or not?
and I am confused about significant tests.
3. Relevance feedback:
about BM25, it is a more complicated feedback model, so does it will verify information need, and how to be based on experience to calculate feedback scores?
1. How to decide the value of trade-off between recall and precision?
2. Evaluate ranked retrieval:
The average values may have different performances, so how we judge the system is stable or not?
and I am confused about significant tests.
3. Relevance feedback:
about BM25, it is a more complicated feedback model, so does it will verify information need, and how to be based on experience to calculate feedback scores?
Friday, February 21, 2014
Week7——Reading Notes(unit7, IIR chapter9)
Chapter 9 Relevance feedback and query expansion:
1. A system can help with query refinement, either fully automatically or with the user in the loop.
2. Relevance feedback:it can also be effective in tracking a user's evolving information need: seeing some documents may lead users to refine their understanding of the information they are seeking, they can easily indicate relevant or non relevant images. It can improve recall and precision.
3. The Rocchio algorithm: it models a way of incorporating relevance feedback information into the vector space model. The underlying theory is the basic to evaluate similarity with relevant documents. The algorithm proposes using the modified query.
4. Naive bayes probabilistic model: use only collection statistics and information about the term distribution within the documents judged relevant. They preserve no memory of the original query.
5. Cases where relevance feedback alone is not sufficient include: misspellings, cross-language information retrieval and mismatch of searcher's vocabulary versus collection vocabulary.
1. A system can help with query refinement, either fully automatically or with the user in the loop.
2. Relevance feedback:it can also be effective in tracking a user's evolving information need: seeing some documents may lead users to refine their understanding of the information they are seeking, they can easily indicate relevant or non relevant images. It can improve recall and precision.
3. The Rocchio algorithm: it models a way of incorporating relevance feedback information into the vector space model. The underlying theory is the basic to evaluate similarity with relevant documents. The algorithm proposes using the modified query.
4. Naive bayes probabilistic model: use only collection statistics and information about the term distribution within the documents judged relevant. They preserve no memory of the original query.
5. Cases where relevance feedback alone is not sufficient include: misspellings, cross-language information retrieval and mismatch of searcher's vocabulary versus collection vocabulary.
Week7——Muddiest Points
Unit6: Evaluation of IR Systems:
1. Important issues in evaluation:
The professor mentioned that in IR system, evaluation is a compare study, so how can we set up the baseline and the judgement of models?
2. Pooling method is cheaper and more powerful, why we choose to pick up top 50 to put together as a pool?
1. Important issues in evaluation:
The professor mentioned that in IR system, evaluation is a compare study, so how can we set up the baseline and the judgement of models?
2. Pooling method is cheaper and more powerful, why we choose to pick up top 50 to put together as a pool?
Thursday, February 13, 2014
Week6——Reading notes(unit 6)
Chapter8 Evaluation in information retrieval
1. Information retrieval system evaluation:
Relevance is assessed relative to an information need, not a query. How can we judge the information need if we not contain all the words in the query.
2. Standard test collections:
the cornfield collection, TREC, GOV2, NTCIR, CLEF, Reuters, 20 Newsgroups. How we decide to choose standards in different situations?
3.Evaluation of unranked retrieval sets:
Precision: the fraction of retrieval documents that are relevant.
Recall: the fraction of relevant documents that are retrieved.
F measure: trades iff precision versus recall, which is the weighted harmonic mean of precision and recall.
4. Evaluation of ranked retrieval results:
Mean Average Precision: provide a single-figure measure of quality across recall levels.
ROC curve: plot the true positive rate or sensitivity against the false positive rate or (1- specificity).
5. Assessing relevance:
It is a time-consuming and expensive process involving human beings.
Pooling: relevance is assessed over a subset of the collection that is formed from the top k documents returned by a number of different IR systems and perhaps other sources such as the results of Boolean keyword searches or documents found by expert searchers in an interactive process.
The relevance of one document is treated as independent of the relevance of other documents in the collection.
1. Information retrieval system evaluation:
Relevance is assessed relative to an information need, not a query. How can we judge the information need if we not contain all the words in the query.
2. Standard test collections:
the cornfield collection, TREC, GOV2, NTCIR, CLEF, Reuters, 20 Newsgroups. How we decide to choose standards in different situations?
3.Evaluation of unranked retrieval sets:
Precision: the fraction of retrieval documents that are relevant.
Recall: the fraction of relevant documents that are retrieved.
F measure: trades iff precision versus recall, which is the weighted harmonic mean of precision and recall.
4. Evaluation of ranked retrieval results:
Mean Average Precision: provide a single-figure measure of quality across recall levels.
ROC curve: plot the true positive rate or sensitivity against the false positive rate or (1- specificity).
5. Assessing relevance:
It is a time-consuming and expensive process involving human beings.
Pooling: relevance is assessed over a subset of the collection that is formed from the top k documents returned by a number of different IR systems and perhaps other sources such as the results of Boolean keyword searches or documents found by expert searchers in an interactive process.
The relevance of one document is treated as independent of the relevance of other documents in the collection.
week6——Muddiest Points
Unit 5: Probabilistic Retrieval Models:
1. About language models:
In IR system, why we only consider unigram, not n-gram? Does it means that one string or one word depending on the string or word before it? How we deal with the new words or new language?
2. About smoothing:
Add one: I think it is only used to eliminate zero probability, some strings or word still have lowest probabilities.
Linear interpolation: how we decide the value of λ?
3. LM vs VS:
LM depends on strings and words, so will it spend lots of space to storage probabilities, and does it not consider the whole document?
1. About language models:
In IR system, why we only consider unigram, not n-gram? Does it means that one string or one word depending on the string or word before it? How we deal with the new words or new language?
2. About smoothing:
Add one: I think it is only used to eliminate zero probability, some strings or word still have lowest probabilities.
Linear interpolation: how we decide the value of λ?
3. LM vs VS:
LM depends on strings and words, so will it spend lots of space to storage probabilities, and does it not consider the whole document?
Subscribe to:
Comments (Atom)