-
Notifications
You must be signed in to change notification settings - Fork 205
PackageRank #1091
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kubaneko
wants to merge
62
commits into
haskell:master
Choose a base branch
from
kubaneko:PageRank
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+517
−17
Open
PackageRank #1091
Changes from all commits
Commits
Show all changes
62 commits
Select commit
Hold shift + click to select a range
40dfc83
started new PageRank branch
kubaneko cbbaeb6
correct the error
kubaneko 164128b
Created PackageRank module and added it to build
kubaneko 54de869
write out ranking criteria
kubaneko 1e2c6c8
started with maintainer number
kubaneko e777bf2
added Upload Feature and got number of maintainers for package
kubaneko 6a45d62
divided rankPackage to pure and IO version
kubaneko 0a6dc00
added benchmark and test info
kubaneko 722103b
added Download Feature
kubaneko 056c833
formatted a bit
kubaneko c0abd3d
used brittany
kubaneko 4c7dba1
added further info about the package
kubaneko 36ef484
added isApp function
kubaneko 53f5104
added Scorer type and started to extract versions
kubaneko 1a157b8
more work on versions
kubaneko 7ab35a9
got versions and partitioned based on status
kubaneko 8d5e68a
separated versions into versionList and verPart
kubaneko 3859803
added last upload times
kubaneko 95e02d3
added freshnessScore
kubaneko c145124
added Download Scorer
kubaneko faa42c2
finished simple temporalScore (rankPackageIO)
kubaneko 64453e8
separated rankIO from temporalScore
kubaneko 3c93212
added pageRank
kubaneko 73a65c9
added versionScore
kubaneko bf38c80
added authorScore
kubaneko bf77ee3
instance Semigroup Scorer
kubaneko a27a0c7
got tarballs and fixed warnings
kubaneko 8931732
extracted documentation length
kubaneko b5ca917
got tarEntries for package and fixed it for documentation
kubaneko 37ce2fb
added codeScore
kubaneko 817559d
replaced some Features by ListFeature
kubaneko 0cd6c96
added some Features to BrowseFeatures - prototype
kubaneko 983606f
Revert "added some Features to BrowseFeatures - prototype"
kubaneko 5147554
Revert "replaced some Features by ListFeature"
kubaneko 22bdc45
changed ListFeature to fit PackageRank
kubaneko 8acc750
changed PackageRank to fit in constructItem
kubaneko 6be2930
integrated PackageRank into ListFeature
kubaneko 8745c69
--no-edit
kubaneko a3bb571
tried to add an column and failed
kubaneko e53968c
switch Doubles for Floats
kubaneko 16d6e67
added the column and redid some packageRank issues
kubaneko f90c797
fixed some basic bugs
kubaneko d878f42
removed Browse/parser changes
kubaneko a7bcef6
Fixed missing titile and changed fixed description
kubaneko 6a887b5
Strict Scorer
kubaneko e881d70
fixed some partial functions
kubaneko b2a80ce
fixed some bugs
kubaneko 3089b6d
fixed a bug
kubaneko b888ccb
retrieves src correctly
kubaneko 4748abd
fixed documentation retrieval
kubaneko 7609a8a
changed the algorithm to match cargo
kubaneko f26effe
prototype for readme parser (collects some info about markdown)
kubaneko 5f38c6b
forgot to add the parser
kubaneko 2ba5071
finished readmeScore
kubaneko a3c81fa
changed documentation parameter to get reasonable output
kubaneko ead8f6b
changed some parameters to reflect hackage
kubaneko 9d4d811
moved PackageRank into PackageList Feature and changed UI so packageR…
kubaneko 2cd996b
added some Exception handling
kubaneko 32995c8
some comments and refactoring
kubaneko 39b28de
test commit
kubaneko 5d354da
fixed dumb errors after rebase, changed memsize of packageitem to 12
kubaneko fc4c527
added reverse Dependencies to PackageRank and tried to scale it for H…
kubaneko File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
{-# LANGUAGE ScopedTypeVariables, FlexibleInstances, MultiParamTypeClasses, ConstraintKinds #-} | ||
module Distribution.Server.Features.PackageList.MStats | ||
( parseM | ||
, sumMStat | ||
, getListsTables | ||
, getCode | ||
, getHCode | ||
, getSections | ||
, MStats(..) | ||
) where | ||
|
||
import Commonmark | ||
import Commonmark.Extensions | ||
import Control.Monad.Identity | ||
import qualified Data.ByteString.Lazy as BS | ||
( ByteString | ||
, toStrict ) | ||
import qualified Data.Text as T | ||
import qualified Data.Text.Encoding as T | ||
import qualified Data.Text.Encoding.Error as T | ||
( lenientDecode ) | ||
|
||
-- parses markdown into statistics needed for readmeScore | ||
parseM :: BS.ByteString -> FilePath -> Either ParseError [MarkdownStats] | ||
parseM md name = runIdentity | ||
(commonmarkWith (pipeTableSpec <> defaultSyntaxSpec) name txt) | ||
where txt = T.decodeUtf8With T.lenientDecode . BS.toStrict $ md | ||
|
||
data MarkdownStats = NotImportant MStats | | ||
HCode MStats | | ||
Code MStats | | ||
Section MStats | | ||
Table Int MStats | -- Int of rows | ||
PText MStats | | ||
List Int MStats -- Int of elements | ||
deriving (Show) | ||
|
||
data MStats = MStats Int Int --number of pictures, number of chars | ||
deriving Show | ||
|
||
instance Monoid MStats where | ||
mempty = MStats 0 0 | ||
|
||
instance Rangeable MStats where | ||
ranged = const id | ||
|
||
instance HasAttributes MStats where | ||
addAttributes = const id | ||
|
||
instance Semigroup MStats where | ||
(MStats a b) <> (MStats c d) = MStats (a + c) (b + d) | ||
|
||
-- Getter functions | ||
|
||
getCode :: [MarkdownStats] -> (Int, Int) -- number of code blocks, size of code | ||
getCode [] = (0, 0) | ||
getCode (Code (MStats codeT _) : xs) = (1, codeT) >< getCode xs | ||
getCode (HCode (MStats codeT _) : xs) = (1, codeT) >< getCode xs | ||
getCode (_ : xs) = getCode xs | ||
|
||
getHCode :: [MarkdownStats] -> (Int, Int) -- number of code blocks, size of code | ||
getHCode [] = (0, 0) | ||
getHCode (HCode (MStats codeT _) : xs) = (1, codeT) >< getHCode xs | ||
getHCode (_ : xs) = getHCode xs | ||
|
||
getSections :: [MarkdownStats] -> Int -- number of code blocks, size of code | ||
getSections [] = 0 | ||
getSections (Section _ : xs) = 1 + getSections xs | ||
getSections (_ : xs) = getSections xs | ||
|
||
sumMStat :: [MarkdownStats] -> MStats | ||
sumMStat [] = mempty | ||
sumMStat (x : xs) = case x of | ||
(NotImportant a) -> a <> sumMStat xs | ||
(Section a) -> a <> sumMStat xs | ||
(List _ a ) -> a <> sumMStat xs | ||
(Table _ a ) -> a <> sumMStat xs | ||
(HCode a ) -> a <> sumMStat xs | ||
(Code a ) -> a <> sumMStat xs | ||
(PText a ) -> a <> sumMStat xs | ||
|
||
getListsTables :: [MarkdownStats] -> Int | ||
getListsTables [] = 0 | ||
getListsTables ((List a _) : ys) = a + getListsTables ys | ||
getListsTables ((Table a _) : ys) = a + getListsTables ys | ||
getListsTables (_ : ys) = getListsTables ys | ||
|
||
-- helper | ||
(><) :: (Int, Int) -> (Int, Int) -> (Int, Int) | ||
(><) (a, b) (c, d) = (a + c, b + d) | ||
|
||
-- INSTANCES | ||
instance Rangeable [MarkdownStats] where | ||
ranged = const id | ||
|
||
instance HasAttributes [MarkdownStats] where | ||
addAttributes = const id | ||
|
||
instance HasPipeTable MStats [MarkdownStats] where | ||
pipeTable _ _ rows = [Table (length rows) (mconcat $ mconcat <$> rows)] | ||
|
||
instance IsInline MStats where | ||
lineBreak = MStats 0 1 | ||
softBreak = MStats 0 1 | ||
str t = MStats 0 (T.length t) | ||
entity t = MStats 0 (T.length t) | ||
escapedChar _ = MStats 0 1 | ||
emph = id | ||
strong = id | ||
link _ _ a = a | ||
image _ _ (MStats a b) = MStats (a + 1) b | ||
code t = MStats 0 (T.length t) | ||
rawInline _ t = MStats 0 (T.length t) | ||
|
||
instance IsBlock MStats [MarkdownStats] where | ||
paragraph a = [PText a] | ||
plain a = [PText a] | ||
thematicBreak = [NotImportant mempty] | ||
blockQuote = id | ||
codeBlock language codeT | language == T.pack "haskell" = [HCode (code codeT)] | ||
| otherwise = [Code (code codeT)] | ||
heading _ a = [Section a] | ||
rawBlock _ _ = [NotImportant mempty] | ||
referenceLinkDefinition _ _ = [NotImportant mempty] | ||
list _ _ l = [List (length l + sumLT l) (mconcat $ sumMStat <$> l)] | ||
where sumLT a = sum (getListsTables <$> a) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.